It's been very slow for me as well--often not loading properly, freezing, and asking me if I want to kill the page or wait.
I subsequently stopped infinite scroll and set the number of posts per page at 5 and now it works fine.
I didn't check CPU/mem load, so I don't know if this is related.
Just thought I'd mention my problem/solution.