It seems that I can't use any other syntax inside headlines. Is this a bug?

No, it's not a bug. It's a limitation of how headlines work in DokuWiki. There are two reasons:

1. From a typographic view it's considered bad style to have any formatting in headlines
2. The headlines are used in the TOC and some other places as well, doing this with markup in between would be technically very complicated and error prone

A more detailed technical explanation is available in this mailing list post.

But what about typographical entities (like em-dash, copyright symbol, curly quotes, etc)?

You can include any special character into the headings. Typographic conversions don't work in headings but adding the resulting characters directly works. You can use the character picker () in the editing toolbar to do that.

But what about illustrating a headline with an icon or making it italic?

General styling can still be done with CSS, including icons, italics, backgrounds and color. See how to style headers with use cases and pictures.

If you really want to insert wiki syntax in the header, try plugin Header2.

The plugin Header2 (and Header3) became unavailable due to a technical reason (See this conversation). You can consider using Wiki-Style Script, which applies wiki syntax of bold, italic, underlined, or monospaced text if they are remaining (such as those in headers and link texts). You can also change color of characters in headers and link texts.