Most jockey wheels dont have bearings* as such, more of a metal bush around the bolt that they spin on. Still needs to be cleaned and greased - just dont be surprised if it isnt the same as the other bearings on the bike.
*I think, technically, the metal bush is referred to as a 'bearing' but they are probably not ball bearings like wheel hubs, headsets, bottom brackets, etc
That's how my jockey wheels are.
When I've cleaned mine in the past, I've just removed the jockey wheels and arm. Soaked them in degreaser and then cleaned the main body with brushes/clothes degreaser etc.