Why Would al Qaeda Return to Afghanistan?

One point that's being over-looked is the question of whether al Qaeda would want to return to Afghanistan if the U.S. left and the Taliban somehow took over the joint again. Consider: however hunted the top leadership of al Qaeda is in Pakistan, they are protected by the Pakistani government from a full-blown American boots-on-the-ground effort to capture and kill them. America must rely on drones, its own intelligence and the good will of Pakistan to take whatever shots it can at al Qaeda.

Were they to return to Afghanistan, could the Taliban afford the same protection? Not even close. There's nothing stopping the U.S. from invading or aggressively bombing Afghanistan at will. The same cannot be said of Pakistan. Therefore it seems that even if Kabul were to once again fall to the Taliban (an outcome that is nowhere near a certainty even if the U.S. reduces its footprint) al Qaeda would be better served staying put in Pakistan. Or am I missing something?