Summary:
Uses `fastfs.readWhile` to build the haste map
- cuts the time needed to build the haste map in half
- we don’t need to allocate memory for all JS files in the tree
- we only read in as much as necessary during startup
- we only read files completely that are part of the bundle
- we will be able to move the transform before dependency extraction
public
Reviewed By: martinbigio
Differential Revision: D2890933
fb-gh-sync-id: 5fef6b53458e8bc95d0251d0bcf16821581a3362
Summary:
public
This adds an option to `Module` (and its callers) that allows to transform code before extracting dependencies. The transform function has to return a promise that resolves to an object with `code`, and optionally `dependencies` and/or `asyncDependencies` properties, if standard dependency extraction cannot be applied
Reviewed By: cpojer
Differential Revision: D2870437
fb-gh-sync-id: 806d24ba16b1693d838a3fa747d82be9dc6ccf00
Summary:
We used to only cache the `dependencies` and `name` of a Module but we were actually accessing two more fields (async dependencies and the `isHaste` field) which meant we were always reading every single file from disk. In D2644383 I noticed that but realized that the Promise was cached, meaning we would read every file once *per instance* and I didn't think about cross-instance reads over time. My initial version added more caching (but also missed the `isHaste` field) but then I got rid of the cache call for `dependencies`. So my change from before didn't make anything worse but it also didn't make anything better. This change now caches everything until the contents of the file actually changes.
public
Reviewed By: martinbigio
Differential Revision: D2831569
fb-gh-sync-id: 74081abc0ce3ca96b4e56c3c9b6d24aa84f7496c
Summary:
There's quite a bit of code scattered around the packager regarding ignoring the `providesModule` Haste pragma in any file that isn't in `react-native`, `react-tools` or `parse`. There is even a (passing) test case.
However, there's an edge case.
Take, for example, `fbjs`. It has a module inside of it called `ErrorUtils`. `react-relay` requires this file normally, in Common.JS style, by doing `require('fbjs/libs/ErrorUtils')`. But when `react-native` attempts to require `ErrorUtils` using the HasteModule format (in it's JavaScript initialization), it resolves the `fbjs` `ErrorUtils` module, instead of RN's `ErrorUtils`.
This happens, it turns out, because when a module is read (in `Module._read`), it's not caring about whether or not it should pay attention to `providesModule`, and is just assigning the `providesModule` value as the id of the module no matter what. Then when `Module.getName` is called, it will always use that `data.id` that was set, thus creating the wrong dependency tree.
This
Closes https://github.com/facebook/react-native/pull/3625
Reviewed By: svcscm
Differential Revision: D2632317
Pulled By: vjeux
fb-gh-sync-id: efd8066eaf6f18fcf79698beab36cab90bf5cd6d
Summary: I think packager on different platform should generate same output if possible. So packager should replace '\\' in module name with '/' on Windows.
Closes https://github.com/facebook/react-native/pull/2813
Reviewed By: @svcscm
Differential Revision: D2458634
Pulled By: @martinbigio
Summary:
Buck (our build system) currently starts multiple packager instances for each target and may build multiple targets in parallel. This means we're paying startup costs and are duplicating the work. This enables us to start one instance of the packager and connect to it via socket to do all the work that needs to be done.
The way this is structured:
1. SocketServer: A server that listens on a socket path that is generated based on the server options
2. SocketClient: Interfaces with the server and exposes the operations that we support as methods
3. SocketInterface: Integration point and responsible for forking off the server
Summary:
We've decided to move the syntax for asynchronously requiring async dependencies. The new syntax works better with promises and therefore withe async/await as well. The new syntax looks like this: `System.import('moduleA').then(moduleA => {...});` or if you're using async/await you could simply do:
let moduleA = await System.import('moduleA');
new moduleA().someFunction();
If you need to require multiple dependencies just do:
Promise
.all([System.import('moduleA'), System.import('moduleB')])
.then((moduleA, moduleB) => {...})
or the equivalent using async/await
Summary:
Instead of using plain objects and having to convert to and from them we just use the `Module` class across the codebase.
This seems cleaner and can enforce the type as opposed to fuzzy objects.
Summary:
Introduce a Bundler capable of generating the layout of modules for a given entry point. The current algorithm is the most trivial we could come up with: (1)it puts all the sync dependencies into the same bundle and (2) each group of async dependencies with all their dependencies into a separate bundle. For async dependencies we do this recursivelly, meaning that async dependencies could have async dependencies which will end up on separate bundles as well.
The output of of the layout is an array of bundles. Each bundle is just an array for now with the dependencies in the order the requires where processed. Using this information we should be able to generate the actual bundles by using the `/path/to/entry/point.bundle` endpoint. We might change the structure of this json in the future, for instance to account for parent/child bundles relationships.
The next step will be to improve this algorithm to avoid repeating quite a bit dependencies across bundles.
Summary:
This is the first step to add support for splitting the JS bundle into multiple ones. This diff adds support for keeping track of the async dependencies each module has. To do so we introduce the following syntax:
require.ensure(['dep1', 'dep2, ..., 'depN'], callback);
Where the callback function is asynchronously invoked once all the indicated modules are loaded.
Internally, the packager keeps track of every set of async dependencies a module has. So for instance if a module looks like this:
require.ensure(['dep1'], () => {...});
require.ensure(['dep2'], () => {...});
the `Module` object will keep track of each set of dependencies separately (because we might want to put them on separate bundles).
Summary:
The cache is only used for JSTransformer at the moment. We're doing IO and some computation to get each module's name, whether is a haste or node module and it's dependencies. This work happens on startup so by caching this value we shouldbe able to reduce the start up time. Lets promote the Cache to the Packager level to be able to use it by any of the components of the packager. For now, on this diff we'll start using it to cache the mentioned fields.
Also we had to introduce the concept of fields in the cache as manually merging the date we had for each path is not possible as we're using promisses all around. With the new API, each field is a promise.
@amasad and I did some manual testing to measure the impact of this change and looks like it's saves 1 second when building the haste map (which represents 50% of the time). Overall this reduces 1 second of start up time which was currently about 8s on my mac book pro.
Summary:
@public
Fixes#773, #1055
The resolver was getting a bit unwieldy because a lot has changed since the initial writing (porting node-haste).
This also splits up a large complex file into the following:
* Makes use of classes: Module, AssetModule, Package, and AssetModule_DEPRECATED (`image!` modules)
* DependencyGraph is lazy for everything that isn't haste modules and packages (need to read ahead of time)
* Lazy makes it fast, easier to reason about, and easier to add new loaders
* Has a centralized filesystem wrapper: fast-fs (ffs)
* ffs is async and lazy for any read operation and sync for directory/file lookup which makes it fast
* we can easily drop in different adapters for ffs to be able to build up the tree: watchman, git ls-files, etc
* use es6 for classes and easier to read promise-based code
Follow up diffs will include:
* Using new types (Module, AssetModule etc) in the rest of the codebase (currently we convert to plain object which is a bit of a hack)
* using watchman to build up the fs
* some caching at the object creation level (we are recreating Modules and Packages many times, we can cache them)
* A plugin system for loaders (e.g. @tadeuzagallo wants to add a native module loader)
Test Plan:
* ./runJestTests.sh react-packager
* ./runJestTests.sh PackagerIntegration
* Export open source and run the e2e test
* reset cache
* ./fbrnios.sh run and click around
