The record-jar Format

The record-jar format is described by the following ABNF ():

record-jar = [encodingSig] [separator] *record record = 1*field separator field = ( field-name field-sep field-body CRLF ) field-name = *character field-sep = *SP ":" *SP field-body = *(continuation 1*character) continuation = ["\"] [[*SP CRLF] 1*SP] separator = [blank-line] *("%%" [comment] CRLF) comment = SP *69(character) character = SP / ASCCHAR / UNICHAR / ESCAPE encodingSig = "%%encoding" field-sep *(ALPHA / DIGIT / "-" / "_") CRLF blank-line = WSP CRLF ; ASCII characters except %x26 (&) and %x5C (\) ASCCHAR = %x21-25 / %x27-5B / %x5D-7E UNICHAR = %x80-%x10FFFF ; Unicode chars ESCAPE = "\" ("\" / "&" / "r" / "n" / "t" ) / "&#x" 2*6HEXDIG ";" The record-jar format consists of character data that forms a sequence of records. Each record is separated from other records by at least one line beginning with the sequence "%%" (%x25.25). Records are made up of one or more fields and a record MAY contain as many or as few fields as are necessary to convey the necessary data. Empty records and blank lines are ignored.A field is a single, logical line of characters from the Universal Character Set (Unicode), comprised of three parts: the field-name, the field-separator, and the field body.The field-name is an identifer. Field-names SHOULD consist only of characters permitted in identifiers according to Unicode Standards Annex #31 (UAX#31) and SHOULD start only with characters with the property ID_Start. Often field-names are further restricted to a sequence of letters and digits from the US-ASCII character set . A field-name SHOULD be treated as case sensitive and MUST NOT contain any spaces. Upper and lowercase letters are often used to visually break up the name, for example using CamelCase. It is a common convention that field names use an initial capital letter, although this is not enforced. The hyphen-minus character ("-", %x2D) MAY be used to separate parts of the name visually, however, it MUST NOT appear at the beginning or end of a field-name.The field separator (field-sep) is the colon character (":", %x3A). The separator MAY be surrounded on either side by any amount of horizontal whitespace (tab or space characters). The normal convention is one space on each side.The field-body contains the data value. Logically, the field-body consists of a single line of text using any combination of characters from the Universal Character Set followed by a CRLF (newline). The carriage return, newline, and tab characters, when they occur in the data value stored in the field-body, are represented by their common backslash escapes ("\r", "\n", and "\t" respectively). See for more information on escape sequences.

Some protocols limit total line length. For example, many Internet plain-text protocols limits lines to 72 total bytes. To accommodate such limits or for readability and presentational purposes, the field-body portion of a field can be split into a multiple-line representation; this is called "folding". Successive lines in the same field-body begin with one or more whitespace characters. When processing the record-jar format, the linear whitespace (including the newline and any preceeding spaces) is consumed by the processor and the two parts of the field-body joined to form a single, logical line. For example:

Eulers-Number : 2.718281828459045235360287471 352662497757247093699959574966967627724076630353547 5945713821785251664274274663919320030599218174135... Note that imposing a line length limit effectively limits the length of the field-name, since the field separator MUST appear on the same line with the field-name and the field-name MUST NOT be folded. Also, when imposing a line length limit, note that some encodings (including the Unicode encodings) can use a variable number of bytes per character or commonly use more than one byte per character. Characters MUST NOT be folded in the middle of a byte sequence. Furthermore, folding SHOULD NOT be done just prior to a combining character (since this will alter the display of characters in the file and might result in unintentional alteration of the file's semantics).In some cases, the field-body contains spaces that are important to the data. To accurately preserve whitespace in the document, an optional line-continuation character (backslash, %x5C) MAY be included to delimit and separate whitespace to be preserved from whitespace that will be removed by the processor. The line-continuation character and any whitespace that follows it (including whitespace at the beginning of the continuing field-body on the next line) MUST be consumed by the processor when reading the file. Whitespace appearing before the line-continuation MUST NOT be consumed. Use of the line continuation character makes the whitespace visible in the file. In other cases, the field-body might contain natural language text, and, while it is readily apparent that many languages use spaces to separate words, others, such as Japanese or Thai, do not. Implementations MAY, in the absence of line continuation characters, replace the continuation sequence (the line break and surrounding whitespace) in a folded line with a single ASCII space (%x20), however, implementations SHOULD just remove the continuation sequence altogether in order to avoid causing unnatural breaks in the text. Here are some examples:

SomeField : This is some running text \ that is continued on several lines \ and which preserves spaces between \ the words. %% AnotherExample: There are three spaces \ between 'spaces' and 'between' in this record. %% SwallowingExample: There are no spaces between \ the numbers one and two in this example 1\ 2. %%Note that entirely blank continuation lines are not permitted. That is, this record is illegal, since the field-body of "SomeText" would be the empty string:

%% SomeText: \ \ \ %%

Comments MAY be included in the body of the record-jar document by placing them at the end of a separator line. The comment MUST be separated by at least one space from the "%%" sequence that introduces the separator. Multiple separators MAY appear between records. Logically this appears to result in records that contain no fields: records containing no fields MUST be ignored by a processor. Folding of comments is not permitted; instead multiple comment lines MUST be used. Comments can not appear in the body of a record. For example:

%% this is a comment. Record: goes here %% %% here is another sequence of comments %% that appear on multiple lines Record: another record %% a final comment %%

By default, a file containing a record-jar archive uses the UTF-8 character encoding (see ). If an application, protocol, or specification permits an encoding other than UTF-8 to be used in the file, it SHOULD also support reading the encoding from the encoding signature. The encoding signature, when present, MUST be the very first line of the file. If the encoding signature is not present, an application or protocol MAY attempt to infer the encoding using other means. Record-jar files SHOULD include an encoding signature, even if one is not required, whenever the application, protocol, or specification permits one.A file that uses the UTF-16 or UTF-32 encoding MAY also include a Byte Order Mark (U+FEFF) as the first sequence of two octets (in the case of UTF-16) or four octets (in the case of UTF-32) in the file, just preceeding the encoding signature.Some applications, protocols, or specifications require that the record-jar file use some other, non-Unicode, legacy character set. In particular, some applications, protocols, or specifications only support the US-ASCII character set ().Here is an example of the encoding signature for the UTF-8 encoding of Unicode:

%%encoding:UTF-8Printable ASCII characters excepting backslash ("\") and ampersand ("&") are represented as themselves.Non-ASCII values MAY be included in a record-jar file in several ways. For portability, the best mechanism is to use escape sequences in the field-body. Exclusive use of escape sequences results in a pure ASCII text file.Non-ASCII characters MAY be represented using the character's Unicode value represented using the Numeric Character Reference format adapted from XML; the sequence "&#x" (%x26.23.78) is followed by the character's Unicode scalar value in hex followed directly by the semi-colon character (";", %x3B). Leading zeroes MAY be omitted. For example, the EURO SIGN is U+20AC and could be represented as "€".Non-ASCII characters MAY also be represented as their associated octet sequence in the file's character encoding. For example, the EURO SIGN would be represented as the byte sequence %xE2.82.AC in UTF-8.The characters for carriage return, newline, and tab when considered as part of the data (and not the file format itself) are represented by the traditional escape sequences "\r" (%x5C.72), "\n" (%x5C.6E), and "\t" (%x5C.74) respectively. The character backslash is represented by "\\" (%x5C.5C), while the ampersand character is represented by "\&" (%x5C.26). A single backslash at the end of a line indicates continuation, as discussed in . Otherwise a single backslash followed by some other character in the data is an error, although a record-jar processor MAY choose to interpret it as a backslash.