Difference between revisions of "Compiler and encoding"
(New page: Category:compiler From 6.7, the compiler has been equiped with a Unicode parser. The core of the parser only accept UTF-8 source code, for simplicity and generality. Before source cod...) |
|||
| Line 1: | Line 1: | ||
[[Category:compiler]] | [[Category:compiler]] | ||
| − | From 6.7, the compiler has been equiped with a Unicode parser. The core of the parser only | + | From 6.7, the compiler has been equiped with a Unicode parser. The core of the parser only accepts UTF-8 source code, for simplicity and generality. Before source code is passed into the core process of parsing, it is preprocessed and converted into UTF-8. |
== Internals == | == Internals == | ||
| + | Abstracted syntax tree now stores STRING_8 as UTF-8 data on each node. There are also different features exporting UTF-8, UTF-32 or the written bytes. Here is an example of how a character é is represented in various levels. | ||
Revision as of 02:34, 30 May 2012
From 6.7, the compiler has been equiped with a Unicode parser. The core of the parser only accepts UTF-8 source code, for simplicity and generality. Before source code is passed into the core process of parsing, it is preprocessed and converted into UTF-8.
Internals
Abstracted syntax tree now stores STRING_8 as UTF-8 data on each node. There are also different features exporting UTF-8, UTF-32 or the written bytes. Here is an example of how a character é is represented in various levels.
