This happens mostly in cooler temps here in south Texas. We tend to forget about it summer months. Anything I can do to stop this. Wall in back of closet appears to be able to remove covering easily.  Should I do that and see if I can add some insulations

Add insulation, increase airflow in the closet, and try to lower the humidity inside the trailer. 

We always left our closet doors open an inch or so to get some circulation in there.  

Yes, DW likes to have it closed. Think that is part of problem. 

Adding insulation to a space not insulated or replacing insulation with some of better quality may help but if you just stuff more insulation material into a previously insulated space you may make the problem work. Insulation like fiberglass works by trapping air and not allowing it to move so putting more into the same space will lower it's ability to insulate. If that wall is already insulated you could improve it by adding another layer of insulation on the inside of that wall but the lack of ventilation is also a major part of the problem. Something like the foil backed styrofoam insulating board could be attached to the inside of the wall. 

Same problem in our bedroom closet.  High rear outer wall.  Drilled two inch holes for these and painted them to match the woodwork.. Two front corners, one high and one low on side, and two along the top.

2 inch round vents

Keep the air warm and moving with this: Closet Heater

Move some conditioned (dehumidified) interior air through the closet - maybe with a super quiet computer fan - drawing air “out” of the closet. This should work in heating or cooling seasons.