The Alternative for Germany party has a strong lead in a regional election, but it is unclear whether it will be able to govern.