Diagnose Internet problems help!

Hey guys, if anyone can help me out by giving me some things to look for to determine the source of my Internet woes it would be much appreciated.

Basically, if I'm playing Xbox, my laptop doesn't even want to connect to the Internet let alone load a page. It helps a bit to plug the machines directly into the router...

Is it an physical problem with the cable Internet, router, modem? What tests can I run to find out?