lukechilds/node - node - Gitea: Git with a cup of tea

14905 Commits

131 Branches

446 Tags

325 MiB

Tree: 7292a1e954

Commit Graph

Author	SHA1	Message	Date
James M Snell	7de59ef925	net: use icu's punycode implementation ICU has a punycode implementation built in. Use it instead of the javascript implementation because it's much faster. PR-URL: https://github.com/nodejs/node/pull/7355 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>	8 years ago
Steven R. Loomis	a30839576c	build: i18n: add icu config options Make "--with-intl=none" the default and add "intl-none" option to vcbuild.bat. If icu data is missing print a warning unless either --download=all or --download=icu is set. If set then automatically download, verify (MD5) and unpack the ICU data if not already available. There's a "list" of URLs being used, but right now only the first is picked up. The logic works something like this: * If there is no directory deps/icu, * If no zip file (currently icu4c-54_1-src.zip), * Download zip file (icu-project.org -> sf.net) * Verify the MD5 sum of the zipfile * If bad, print error and exit * Unpack the zipfile into deps/icu * If deps/icu now exists, use it, else fail with help text Add the configuration option "--with-icu-source=..." Usage: * --with-icu-source=/path/to/my/other/icu * --with-icu-source=/path/to/icu54.zip * --with-icu-source=/path/to/icu54.tgz * --with-icu-source=http://example.com/icu54.tar.bz2 Add the configuration option "--with-icu-locals=...". Allows choosing which locales are used in the "small-icu" case. Example: configure --with-intl=small-icu --with-icu-locales=tlh,grc,nl (Also note that as of this writing, neither Klingon nor Ancient Greek are in upstream CLDR data. Serving suggestion only.) Don't use hard coded ../../out paths on windows. This was suggested by @misterdjules as it causes test failures. With this fix, "out" is no longer created on windows and the following can run properly: python tools/test.py simple Reduce space by about 1MB with ICU 54 (over without this patch). Also trims a few other source files, but only conditional on the exact ICU version used. This is to future-proof - a file that is unneeded now may be needed in future ICUs. Also: * Update distclean to remove icu related files * Refactor some code into tools/configure.d/nodedownload.py * Update docs * Add test PR-URL: https://github.com/joyent/node/pull/8719 Fixes: https://github.com/joyent/node/issues/7676#issuecomment-64704230 [trev.norris@gmail.com small change to test's whitespace and logic] Signed-off-by: Trevor Norris <trev.norris@gmail.com>	10 years ago
Steven R. Loomis	ac2857b12c	build, i18n: improve Intl build, add "--with-intl" The two main goals of this change are: - To make it easier to build the Intl option using ICU (particularly, using a newer ICU than v8/Chromium's version) - To enable a much smaller ICU build with only English support The goal here is to get node.js binaries built this way by default so that the Intl API can be used. Additional data can be added at execution time (see Readme and wiki) More details are at https://github.com/joyent/node/pull/7719 In particular, this change adds the "--with-intl=" configure option to provide more ways of building "Intl": - "full-icu" picks up an ICU from deps/icu - "small-icu" is similar, but builds only English - "system-icu" uses pkg-config to find an installed ICU - "none" does nothing (no Intl) For Windows builds, the "full-icu" or "small-icu" options are added to vcbuild.bat. Note that the existing "--with-icu-path" option is not removed from configure, but may not be used alongside the new option. Wiki changes have already been made on https://github.com/joyent/node/wiki/Installation and a new page created at https://github.com/joyent/node/wiki/Intl (marked as provisional until this change lands.) Summary of changes: * README.md : doc updates * .gitignore : added "deps/icu" as this is the location where ICU is unpacked to. * Makefile : added the tools/icu/* files to cpplint, but excluded a problematic file. * configure : added the "--with-intl" option mentioned above. Calculate at config time the list of ICU source files to use and data packaging options. * node.gyp : add the new files src/node_i18n.cc/.h as well as ICU linkage. * src/node.cc : add call into node::i18n::InitializeICUDirectory(icu_data_dir) as well as new --icu-data-dir option and NODE_ICU_DATA env variable to configure ICU data loading. This loading is only relevant in the "small" configuration. * src/node_i18n.cc : new source file for the above Initialize.. function, to setup ICU as needed. * tools/icu : new directory with some tools needed for this build. * tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new ways, both on unix/mac and windows. * tools/icu/icu-system.gyp : new .gyp file to build node against a pkg-config detected ICU. * tools/icu/icu_small.json : new config file for the "English-only" small build. * tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the above .json file. * tools/icu/iculslocs.cc : new tool for repairing ICU data manifests after trim operation. * tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker. * vcbuild.bat : added small-icu and full-icu options, to call into configure. * Fixed toolset dependencies, see https://github.com/joyent/node/pull/7719#issuecomment-54641687 Note that because of a bug in gyp {CC,CXX}_host must also be set. Otherwise gcc/g++ will be used by default for part of the build. Reviewed-by: Trevor Norris <trev.norris@gmail.com> Reviewed-by: Fedor Indutny <fedor@indutny.com>	10 years ago

Author

SHA1

Message

Date

James M Snell

7de59ef925

net: use icu's punycode implementation

ICU has a punycode implementation built in. Use it instead of the
javascript implementation because it's much faster.

PR-URL: https://github.com/nodejs/node/pull/7355
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

8 years ago

Steven R. Loomis

a30839576c

build: i18n: add icu config options

Make "--with-intl=none" the default and add "intl-none" option to
vcbuild.bat.

If icu data is missing print a warning unless either --download=all or
--download=icu is set. If set then automatically download, verify (MD5)
and unpack the ICU data if not already available.

There's a "list" of URLs being used, but right now only the first is
picked up. The logic works something like this:

* If there is no directory deps/icu,
  * If no zip file (currently icu4c-54_1-src.zip),
    * Download zip file (icu-project.org -> sf.net)
  * Verify the MD5 sum of the zipfile
    * If bad, print error and exit
  * Unpack the zipfile into deps/icu
* If deps/icu now exists, use it, else fail with help text

Add the configuration option "--with-icu-source=..."

Usage:
  * --with-icu-source=/path/to/my/other/icu
  * --with-icu-source=/path/to/icu54.zip
  * --with-icu-source=/path/to/icu54.tgz
  * --with-icu-source=http://example.com/icu54.tar.bz2

Add the configuration option "--with-icu-locals=...".  Allows choosing
which locales are used in the "small-icu" case.

Example:
    configure --with-intl=small-icu --with-icu-locales=tlh,grc,nl

(Also note that as of this writing, neither Klingon nor Ancient Greek
are in upstream CLDR data. Serving suggestion only.)

Don't use hard coded ../../out paths on windows. This was suggested by
@misterdjules as it causes test failures.  With this fix, "out" is no
longer created on windows and the following can run properly:

    python tools/test.py simple

Reduce space by about 1MB with ICU 54 (over without this patch). Also
trims a few other source files, but only conditional on the exact ICU
version used. This is to future-proof - a file that is unneeded now may
be needed in future ICUs.

Also:
  * Update distclean to remove icu related files
  * Refactor some code into tools/configure.d/nodedownload.py
  * Update docs
  * Add test

PR-URL: https://github.com/joyent/node/pull/8719
Fixes: https://github.com/joyent/node/issues/7676#issuecomment-64704230
[trev.norris@gmail.com small change to test's whitespace and logic]
Signed-off-by: Trevor Norris <trev.norris@gmail.com>

10 years ago

Steven R. Loomis

ac2857b12c

build, i18n: improve Intl build, add "--with-intl"

The two main goals of this change are:
 - To make it easier to build the Intl option using ICU (particularly,
   using a newer ICU than v8/Chromium's version)
 - To enable a much smaller ICU build with only English support The goal
   here is to get node.js binaries built this way by default so that the
   Intl API can be used. Additional data can be added at execution time
   (see Readme and wiki)

More details are at https://github.com/joyent/node/pull/7719

In particular, this change adds the "--with-intl=" configure option to
provide more ways of building "Intl":
 - "full-icu" picks up an ICU from deps/icu
 - "small-icu" is similar, but builds only English
 - "system-icu" uses pkg-config to find an installed ICU
 - "none" does nothing (no Intl)

For Windows builds, the "full-icu" or "small-icu" options are added to
vcbuild.bat.

Note that the existing "--with-icu-path" option is not removed from
configure, but may not be used alongside the new option.

Wiki changes have already been made on
 https://github.com/joyent/node/wiki/Installation
and a new page created at
 https://github.com/joyent/node/wiki/Intl
(marked as provisional until this change lands.)

Summary of changes:

* README.md : doc updates

* .gitignore : added "deps/icu" as this is the location where ICU is
  unpacked to.

* Makefile : added the tools/icu/* files to cpplint, but excluded a
  problematic file.

* configure : added the "--with-intl" option mentioned above.
  Calculate at config time the list of ICU source files to use and data
  packaging options.

* node.gyp : add the new files src/node_i18n.cc/.h as well as ICU
  linkage.

* src/node.cc : add call into
  node::i18n::InitializeICUDirectory(icu_data_dir) as well as new
  --icu-data-dir option and NODE_ICU_DATA env variable to configure ICU
  data loading. This loading is only relevant in the "small"
  configuration.

* src/node_i18n.cc : new source file for the above Initialize..
  function, to setup ICU as needed.

* tools/icu : new directory with some tools needed for this build.

* tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new
  ways, both on unix/mac and windows.

* tools/icu/icu-system.gyp : new .gyp file to build node against a
  pkg-config detected ICU.

* tools/icu/icu_small.json : new config file for the "English-only" small
  build.

* tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the
  above .json file.

* tools/icu/iculslocs.cc : new tool for repairing ICU data manifests
  after trim operation.

* tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker.

* vcbuild.bat : added small-icu and full-icu options, to call into
  configure.

* Fixed toolset dependencies, see
  https://github.com/joyent/node/pull/7719#issuecomment-54641687

Note that because of a bug in gyp {CC,CXX}_host must also be set.
Otherwise gcc/g++ will be used by default for part of the build.

Reviewed-by: Trevor Norris <trev.norris@gmail.com>
Reviewed-by: Fedor Indutny <fedor@indutny.com>

10 years ago

3 Commits (7292a1e954e0db348ff78c704df9e105ba5667ad)