There are at least seven candidates in the running for the top job, with the two main contenders coming from Italy.
Whoever wins will play a crucial role in Brexit, as the European Parliament is mandated to have the final say on any final deal between the UK and the EU.
A secret ballot will be held among 751 MEPs in the French city of Strasbourg.
The frontrunner is Italian politician Antonio Tajani, a former spokesman for Italy's scandal-prone former prime minister Silvio Berlusconi.
Mr Tajani has previously served as European commissioner and is the candidate of the centre-right European People's Party, the largest group in the parliament.
Another Italian, socialist Gianni Pittella, is also in contention.
Throughout most of the parliament's history, the EPP and the Progressive Alliance of Socialists & Democrats have rotated the presidency under a "grand coalition".
However, that arrangement has recently fallen apart amid disagreements over who should succeed Mr Schulz, who is returning to domestic German politics after nearly five years as EU legislative leader.
Therefore it is the smaller, populist parties like UKIP and France's National Front who could decide which candidate wins.
Liberal and former Belgian prime minister Guy Verhofstadt, who is also the parliament's Brexit negotiator, had hoped he might have a chance given the spilt, but that now looks unlikely.
Green candidate Jean Lambert is from the UK, though she has virtually no chance of winning and is unlikely to be a MEP by the time of the next elections in 2019.
Today's vote could go on for up to four rounds as an absolute majority in the European Parliament is needed to win the presidency.
Whoever wins, while not formally part of the Brexit negotiations, will lead a parliament which expects to be kept informed and will hold sway as it could potentially veto any deal.