All insulation wears out, but gloves are probably the fastest to go. They all try and create a lofted insulated space to regulate heat transfer. Some are better than others, but eventually the air will always go out of the balloon. Fluffing can help revive some loft, but it works better in more open soft goods like a jacket.
Synthetic fleece is going to be your toughest and most durable. But the fleece will eventually pill or clump and finally just wear thin from the abrasive action of rubbing up to your hand.
Synthetic puffers like PrimaLoft work great on the back side of your hand. The light and airy space they create would just get crushed in your palm. It also does no absorb water, which is key when your hands sweat. However, synthetic lofted insulation is short lived. The delicate fibers creating the airy matrix will break, twist together, or clump from oils left over from your sweat.
Fleece palm and synthetic lofted backer is the most popular option and usually the best bang for your buck. But they won’t last forever and will get progressively colder.
Down in gloves is warmest and the down feathers are more durable then synthetic lofted fibers. But the downsides of down are just exaggerated in gloves. Down, even treated, will absorb sweat and loose loft, and then hold on to all the oils and dirt it gets and then be hardest to clean. You want down when short term warmth is needed above all else, but it has too many drawbacks in gloves.
The best you can do is use a good pair of glove liners that you can replace. Then invest in gloves with replaceable liners. They will wear, but be cheaper than a new glove.
The problem with glove insulation packing out so easily is why cheap brands like Kinco and Flylow are so popular. You don’t feel bad just getting a new pair.