1. Prepare the cream. Pour half a cup milk into a bowl with the egg yolks, sugar, and cornstarch. Pour remaining milk into a pot with the vanilla extract and bring to a boil.
2. Pour some of the hot milk into the egg-yolk mixture and whisk well. Add the rest of the milk and whisk well. Return to the pot and cook over medium heat for two to three minutes, whisking vigorously, until slightly thickened. Remove from heat. Cover the cream with plastic wrap and cool slightly.
3. Meanwhile, in the bowl of an electric mixer, beat heavy cream or whipped topping until stiff. Fold milk-egg mixture into cream. Transfer to a piping bag fitted with a small star tip (or no tip).
4. Bake and assemble the cream puffs: Preheat the oven to 375 degrees Fahrenheit (190 degrees Celsius).
5. Transfer the cream puffs as directed, but instead of piping individual puffs, pipe rows of four small adjacent cream puffs on a baking sheet and bake as directed.
6. Cut the cream puffs in half and fill with diplomat cream using a piping bag. Sprinkle with confectioners’ sugar and serve. The cream puffs can be stored in the freezer for a month, without the filling.