PHP was initially a HTML preprocessor, and the interprreter only executes code inside <?php ?> tags; the rest is left untouched. The way you are using it now, php would just print the text phpinfo() back to you.
But it doesn't do so because it is waiting for the rest of the page to be input, and will delay execution until it reaches "end of file". On Windows this means you have to enter Ctrl-Z on a line by itself.
That, and your input is missing the "end of statement" marker – the ;.
<?php
phpinfo();
Ctrl-Z Enter
If you want to use "interactive" mode as in Python or Ruby's irb, you need to run php -a, which will execute every line immediately.
You will still need <?php, however, and it will still immediately die on syntax errors, so (these were fixed in 5.4)
I suggest you to use a text editor instead – write some code, save it to test.php, then run php test.php from command line. Edit, fix bugs, run php test.php again.