Mercurial > minori

annotate dep/utf8proc/README.md @ 360:6ca952f6a95c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
CI/linux: need desktop-file-utils package
author Paper <paper@paper.us.eu.org>
date Mon, 15 Jul 2024 01:09:52 -0400
parents 1faa72660932
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
343
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
1 # utf8proc
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
2 [![CI](https://github.com/NanoComp/meep/actions/workflows/build-ci.yml/badge.svg)](https://github.com/JuliaStrings/utf8proc/actions/workflows/build-ci.yml)
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
3 [![AppVeyor status](https://ci.appveyor.com/api/projects/status/ivaa0v6ikxrmm5r6?svg=true)](https://ci.appveyor.com/project/StevenGJohnson/utf8proc)
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
4
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
5 [utf8proc](http://juliastrings.github.io/utf8proc/) is a small, clean C
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
6 library that provides Unicode normalization, case-folding, and other
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
7 operations for data in the [UTF-8
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
8 encoding](http://en.wikipedia.org/wiki/UTF-8). It was [initially
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
9 developed](http://www.public-software-group.org/utf8proc) by Jan
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
10 Behrens and the rest of the [Public Software
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
11 Group](http://www.public-software-group.org/), who deserve *nearly all
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
12 of the credit* for this package. With the blessing of the Public
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
13 Software Group, the [Julia developers](http://julialang.org/) have
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
14 taken over development of utf8proc, since the original developers have
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
15 moved to other projects.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
16
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
17 (utf8proc is used for basic Unicode
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
18 support in the [Julia language](http://julialang.org/), and the Julia
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
19 developers became involved because they wanted to add Unicode 7 support and other features.)
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
20
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
21 (The original utf8proc package also includes Ruby and PostgreSQL plug-ins.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
22 We removed those from utf8proc in order to focus exclusively on the C
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
23 library for the time being, but plan to add them back in or release them as separate packages.)
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
24
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
25 The utf8proc package is licensed under the
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
26 free/open-source [MIT "expat"
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
27 license](http://opensource.org/licenses/MIT) (plus certain Unicode
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
28 data governed by the similarly permissive [Unicode data
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
29 license](http://www.unicode.org/copyright.html#Exhibit1)); please see
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
30 the included `LICENSE.md` file for more detailed information.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
31
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
32 ## Quick Start
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
33
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
34 Typical users should download a [utf8proc release](http://juliastrings.github.io/utf8proc/releases/) rather than cloning directly from github.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
35
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
36 For compilation of the C library, run `make`. You can also install the library and header file with `make install` (by default into `/usr/local/lib` and `/usr/local/bin`, but this can be changed by `make prefix=/some/dir`). `make check` runs some tests, and `make clean` deletes all of the generated files.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
37
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
38 Alternatively, you can compile with `cmake`, e.g. by
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
39 ```sh
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
40 mkdir build
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
41 cmake -S . -B build
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
42 cmake --build build
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
43 ```
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
44
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
45 ### Using other compilers
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
46 The included `Makefile` supports GNU/Linux flavors and MacOS with `gcc`-like compilers; Windows users will typically use `cmake`.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
47
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
48 For other Unix-like systems and other compilers, you may need to pass modified settings to `make` in order to use the correct compilation flags for building shared libraries on your system.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
49
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
50 For HP-UX with HP's `aCC` compiler and GNU Make (installed as `gmake`), you can compile with
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
51 ```
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
52 gmake CC=/opt/aCC/bin/aCC CFLAGS="+O2" PICFLAG="+z" C99FLAG="-Ae" WCFLAGS="+w" LDFLAG_SHARED="-b" SOFLAG="-Wl,+h"
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
53 ```
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
54 To run `gmake install` you will need GNU coreutils for the `install` command, and you may want to pass `prefix=/opt libdir=/opt/lib/hpux32` or similar to change the installation location.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
55
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
56 ## General Information
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
57
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
58 The C library is found in this directory after successful compilation
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
59 and is named `libutf8proc.a` (for the static library) and
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
60 `libutf8proc.so` (for the dynamic library).
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
61
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
62 The Unicode version supported is 15.1.0.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
63
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
64 For Unicode normalizations, the following options are used:
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
65
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
66 * Normalization Form C: `STABLE`, `COMPOSE`
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
67 * Normalization Form D: `STABLE`, `DECOMPOSE`
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
68 * Normalization Form KC: `STABLE`, `COMPOSE`, `COMPAT`
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
69 * Normalization Form KD: `STABLE`, `DECOMPOSE`, `COMPAT`
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
70
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
71 ## C Library
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
72
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
73 The documentation for the C library is found in the `utf8proc.h` header file.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
74 `utf8proc_map` is function you will most likely be using for mapping UTF-8
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
75 strings, unless you want to allocate memory yourself.
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
76
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
77 ## To Do
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
78
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
79 See the Github [issues list](https://github.com/JuliaLang/utf8proc/issues).
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
80
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
81 ## Contact
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
82
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
83 Bug reports, feature requests, and other queries can be filed at
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
84 the [utf8proc issues page on Github](https://github.com/JuliaLang/utf8proc/issues).
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
85
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
86 ## See also
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
87
1faa72660932 *: transfer back to cmake from autotools
Paper <paper@paper.us.eu.org>
parents:
diff changeset
88 An independent Lua translation of this library, [lua-mojibake](https://github.com/differentprogramming/lua-mojibake), is also available.