The braces { } are for normal arguments and the parentheses ( ) are only used for picture macros to specify coordinates and such. Another set are the brackets [ ] for optional arguments.
Normal LaTeX macros always await braces, which are the normal argument and grouping characters. You can't define new macros awaiting parentheses using \newcommand and the similar LaTeX macros.
There is, however, the possibility to define arbitrary patterns for macro with the lower-level plainTeX macro \def.
This pattern is called a parameter text. If this text doesn't follow the macro an error is generated. The syntax for \def is:
\def<macro><parameter text>{<macro content>}
The parameter text must contain the used arguments #1, #2 and can also contain some other text like parentheses.
An example of an macro which awaits a ( ) would be:
\def\myvector(#1,#2){\dosomething{#1}{#2}}
The macro \myvector awaits a text (<something>,<something else>) direct afterwards, where <something> is stored in #1 and <something else> is stored in #2.
See also this question for an discussion about the differences of \def and \newcommand.
One important difference between { } and ( ) or [ ] is:
The { } can be nested, but the others can't. Because { } are handled by the lower-level system every { increases an internal counter which is then decreased when } is found. This way every { is matched with its }. However in \myvector above the first ) will close the parameter text independent of the number of (. The same is true for optional arguments which can't include [ ] for the same reason. One way to avoid this is to surround the inner text with { } which must be matched first before the rest of parameter text can be found. In other words the parameter text scanner is taking everything in braces as one element, so they effective hide their content from the scanner.
Example:
\somemacrowithoptionalargument[ \someother[opt]{...} ]{ ... }
will take \someother[opt as optional argument and the text token as mandatory one. The correct way to write it is:
\somemacrowithoptionalargument[ {\someother[opt]{...}} ]{ ... }
\expdoesn't take any arguments. It just prints “exp” (in upright font). Then\exp(x)prints “exp(x)” and\exp xprints “exp x” (a better example would be\sinwhere both forms are commonly used in math). – Caramdir Feb 16 '11 at 00:42