It would have been a bigger project, but ...

Sometimes the inner drywall walls can be removed, allowing the closet interior to be expanded/reconfigured into the newly exposed wall cavities. Exact results and possibilities will depend on what kind of structure detail is found within the walls.

Depending on the situation it is possible to just paint the inside of the now exposed drywall surfaces (which is the backside of the 'exterior' drywall surface).

In one case I installed vertical plywood panels (carefully crafted as needed to fit) to create a new closet shape, then added a drywall layer directly onto the plywood. In another situation I just painted the plywood surfaces and then attached the shelf hardware with wood screws, no drywall plugs needed.

Interior wall cavities behind drywall (for stick built houses) are typically 3.5" deep, sometimes that is enough extra space to make the effort worthwhile.

For the closet shown here, it is possible to have the shelf front edges extend into the doorway space, such that the closed closet door would be just shy of touching the extended shelves.