There are 3 basic types of permalinks, aka urls, that you can have on your WordPress blog. They are “ugly” permalinks, almost pretty permalinks, and pretty permalinks.
Ugly permalinks are the most basic, and they look like this: http://www.mywebsite.com/?page_id=3 (with 3 being the number of the blog post, which of course changes with the next blog post).
Almost Pretty permalinks look just like a Pretty permalink, except for the addition of “index.php” between the domain url and the pagename. http://www.mywebsite.com/index.php/pagename
A Pretty permalinks often looks like this: http://www.mywebsite.com/pagename
There are variations of the permalink that you may want to use, which includes things like the category, the date, the month and date, etc.
Because the process for changing from ugly to pretty permalinks is not the same on Windows as it is on Apache, you generally will end up with the “almost” pretty permalinks on a Windows server.
Jamie Moynahan has a tutorial on setting this up on pipeten.info and you can find more info on wordpress.org in the WordPress codex.