RDJ had an interview...

...and he talked about his future in the MMU.

While I honestly think there should (eventually) be an Iron Man 4, Robert Downey Jr. doesn't have to be in it.

Go ahead and say what you want, but there's precedent.

Look at James Bond. There have been six (official) actors in the role, and moviegoers all over the world have accepted it. Granted, the first casting change - Connery to Lazenby - didn't go over so well...so not well that Connery returned for Diamonds Are Forever, but everyone wanted Brosnan to break his contract with Remington Steele and take over the franchise.

So there's no reason why Tony Stark can't also be cast again...but he'll never re-cast.