Place potatoes in a large pot. Cover with water and bring to a boil. Boil until potatoes are tender, about 20 to 25 minutes. Drain and let potatoes cool a bit.
In a small bowl, whisk together mayonnaise, sugar, and yellow mustard.
In a large bowl, combine potatoes, celery, onion, pickles, and pepper. Add in mayonnaise mixture and gently fold to combine. Season with salt and pepper to taste.
Although this salad can be served fresh and warm, it can also be made up to a day ahead. Serve cold or bring to room temperature.