Personally, when learning Lisp, syntax wasn't a big issue. While the infix syntax of most languages is fairly intuitive, the rest of syntax (of e.g. C++ or Perl) is scary compared to Lisp's parens. (Although perhaps what puts people off is the frightening simplicity of Lisp's lack of syntax after using a conventional language.)
One the things that proved difficult about learning Lisp was choosing an implementation. There is no canonical implementation and as such I had to try several out before I learned which ones I enjoyed using. This survey of CL implementations (http://common-lisp.net/~dlw/LispSurvey.html) helped me when deciding which implementations to try. I recommend both CLISP (http://clisp.cons.org/) and SBCL (http://www.sbcl.org/), but that is just personal preference.
Also, the IDE was a difficult issue. I eventually settled on emacs with slime (http://common-lisp.net/project/slime/), although I have occasionally used Cusp, a Lisp plugin for Eclipse (http://bitfauna.com/projects/cusp/index.html). The main problem here was a lack of (thourough, easy to follow, up to date) instructions. Over the last couple of years I have gradually found good instructions on slime, but I can't seem to find any of them right now...
I wasn't saying that the prefix syntax was an issue, just that it hampered popularity. Perception ≠ reality, after all. I too quite like the prefix syntax. I still slightly miss the simplicity of some of the Ruby code that I wrote, but the prefix syntax always wins me over; its benefits (macros) outweigh the minor downsides. Especially with []s and ssyntax.
And thanks for the CL thoughts! Regarding IDEs, I myself have ended up working just from the Mac OS X text editor TextMate (using Visor, which puts Terminal on a hotkey), and haven't found it problematic.
