Yes it is safe to use a filler or adhesive e.g. Painters mate. If the holes are quite large you may need to insert and screw a piece of timber or board in the back to stop the filler falling into the hole. This can also be done with a mesh or similar to enable the filler to stay in place. You will likely have to fill the hole in 2 stages as it can fall out if to heavy a coat is applied in one go.
Hope this helps, if I can be of any more assistance please ask.