Trim the ends of the baguette and cut into small pieces. Add the baguette pieces, peeled garlic cloves, orange juice, orange zest, and the egg yolks in a tall mixing bowl. Puree with an immersion blender. Gradually incorporate the oil while mixing, until a creamy and thick aioli has formed.
Season to taste with salt and pepper. Slice the remaining baguette into slices. Serve the orange aioli with the baguette slices for dipping.