This summer has been challenging for the fruit trees in my garden, with intense heat and a lack of rain causing poor cropping, premature ripening and pre-harvest drop.

While my dessert and cooking apples have produced just a handful of stunted, crab-apple-sized fruit, and my quince is bare, the Victoria plum growing next to the greenhouse is the exception. Its branches are so heavily laden that several have snapped under the weight.

For the past week, I’ve been plucking and eating the plums straight from the tree. Occasionally, as I’ve bitten into them, the soft, honeyed flesh has given way to the bitter taste and gritty texture of coffee grounds. I assumed the woody endocarp that protects the seed had started to break down, perhaps due to environmental stress. It wasn’t until I cut one open and pulled out the pit, revealing a small, vivid pink caterpillar surrounded by crumbly brown pellets, that I realised I had been ingesting frass – insect excrement.

The culprits were the larvae of the plum fruit moth (Grapholita funebrana), a drab, greyish-brown species that is locally common in southern England. In mid-summer, females lay individual eggs on developing fruit, and when the creamy-white, brown-headed caterpillars hatch, they burrow in, feasting on the flesh around the stone. As they grow and mature, their bodies flush pink.

From a distance, the dusky purple plums looked flawless. Upon closer inspection, I discovered that affected fruit were slightly misshapen, and there was often a globule of clear, sticky sap oozing from the barely visible entry hole – a defence mechanism known as gummosis, triggered when the caterpillar bores into the skin.

I usually leave fallen fruit for the blackbirds, peacock butterflies and wasps. But to prevent the caterpillars from emerging, pupating and perpetuating the cycle, the advice is to remove all dropped and damaged plums, disposing of them well away from the tree.

I salvaged what I could to make jam, consigning the infested scraps to the compost bin. The following morning, I came into the kitchen to find a squirming mass of caterpillars had migrated out of the fruit and on to the bin lid.

• Under the Changing Skies: The Best of the Guardian’s Country Diary, 2018-2024, is available now at guardianbookshop.com