For my conures and my cockatiel which are about the same size as your caique, I feed them fresh food twice a day and they always have pellets in their cage (right now, laefber's cockatiel size but I also have fed Zupreem naturals and am going to try Mazuri)
My conures are on a breeder diet, so they're getting a bit more calcium rich foods and a bit higher fat/protein, but my cockatiel is on a regular diet.
I took a cheap seed brand without any pellets, soak them overnight and rinse every few hours for a day or two until they start to sprout. About a teaspoon of that, with a pinch of flax seed, sesame seeds, pepitas, or crushed almonds which equals about another teaspoon (I eyeball it and toss in whatever I have on hand)
I chop apple, kale, a little spinach, banana, sometimes fresh banana or banana chips, dried unsulfered coconut, a few raisins or cranberries (dried or fresh) and run them through my food processer for a fine chop.
A little cooked pasta, brown or wild rice, quinoa, white millet, oats, barley or legumes like lentils or split peas, again, whatever I have on hand. I think I give about 3 tablespoons of fresh food per bird per serving. Sometimes it's all gone, sometimes it's all on the cage floor.
Variety is the most important thing. Your caique is probably pretty high energy, and if he's out all day, playing, or flying which is great exercise, you can take a little more liberty with the protein and fat content because he'll burn it off. I aim for about 20% seeds and nuts, 40% fresh fruit and vegetables, 20% grains and legumes, and 20% pellets.
I'm by no means an expert and your results may vary, consider getting a scale to weigh your bird so you can make sure he's not gaining or losing too much weight. A little loss is normal during a diet change, but too much means they're not eating. Shop within the season too. Like right now, I've been getting plenty of squash, pumpkin and cranberries, so those are getting incorporated into the diet. I always have apples and bananas at home since I eat them every day myself.
I don't give supplements, except some extra calcium to my breeders and my "dusty" birds, like my Grey and Cockatiel, and that's maybe only once or twice a week at most and it's much less than the recommended, since I do give eggshell and provide a cuttlebone. Check out some of the recipes in this section, maybe even lay out a paper plate with some different foods and see what your caique goes for!