| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This changes the way bitbake server works quite radically. Now, the
server is always a process based server with the option of starting
an XMLRPC listener on a specific inferface/port.
Behind the scenes this is done with a "bitbake.sock" file alongside
the bitbake.lock file. If we can obtain the lock, we know we need
to start a server. The server always listens on the socket and UIs
can then connect to this. UIs connect by sending a set of three file
descriptors over the domain socket, one for sending commands, one for
receiving command results and the other for receiving events.
These changes meant we can throw away all the horrid server abstraction
code, the plugable transport option to bitbake and the code becomes
much more readable and debuggable. It also likely removes a ton of
ways you could hang the UI/cooker in weird ways due to all the race
conditions that existed with previous processes.
Changes:
* The foreground option for bitbake-server was dropped. Just tail
the log if you really want this, the codepaths were complicated enough
without adding one for this.
* BBSERVER="autodetect" was dropped. The server will autostart and
autoconnect in process mode. You have to specify an xmlrpc server
address since that can't be autodetected. I can't see a use case
for autodetect now.
* The transport/servetype option to bitbake was dropped.
* A BB_SERVER_TIMEOUT variable is added which allows the server
to stay resident for a period of time after the last client
disconnects before unloading. This is used if the -T/--idle-timeout
option is not passed to bitbake.
This change is invasive and may well introduce new issues however I
believe the codebase is in a much better position for further
development and debugging.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To quote Paul:
Not that long ago we added a prefix to logged messages to allow us to see
where the message has been generated (recipe / task). This is undoubtedly
useful for errors and warnings, however, I'm not sure it's really appropriate
for bb.plain(). As an example, see the output for -c listtasks now:
...
NOTE: Executing RunQueue Tasks
nodejs-native-4.5.0-r0 do_listtasks: do_addto_recipe_sysroot
nodejs-native-4.5.0-r0 do_listtasks: do_build Default task for a recipe - depends on all other normal tasks required to 'build' a recipe
nodejs-native-4.5.0-r0 do_listtasks: do_checklicense
nodejs-native-4.5.0-r0 do_listtasks: do_checklicenseall
nodejs-native-4.5.0-r0 do_listtasks: do_checkpkg
...
This patch excludes PLAIN messages from this prefixing making the log output
neater.
[YOCTO #11457]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When heartbeat event support was added it was only added to process.py. Add
it to server/xmlrpc too. There is duplicated code however since we're likely
to combine the server abstractions soon its not worth worrying about now.
This ensures the backends have the same event support.
[YOCTO #10741]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Messages printed when no UI is connected (e.g. memres) are currently lost.
Use the existing queue mechanism to queue these until a UI attaches, then
replay them. This isn't ideal but better than the current situation of
losing them entirely.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
You might think Queue.Queue.get(True, 0) would return an event immediately
if present and otherwise return. It doesn't, it immediately "times out"
and returns with nothing from the queue.
The behaviour we want is not to wait but return anything present which is
what .get(False) does so map to this.
This fixes some odd behaviour observed in some of the tinfoil selftests.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The test could break in a variety of ways:
a) If BB_HEARTBEAT_EVENT was less than ~0.25 it would hang indefinitely
b) The mask is set after draining the event queue meaning a heartbeat event
could have happened
c) The test exits once it sees the events it wants, it doesn't check for
spurious events such as heartbeats which shouldn't have occured.
d) The hardcoded delay of 0.25 is nasty and shouldn't be needed.
I found a bitbake bug and fixed that meaning we don't need the delay any
more which fixes d). That means a) is no longer an issue either.
We now set the mask, then drain the queue meaning no spurious events should
be able to sneak in. The test is also tweaked to wait for 5s in total to
ensure spurious events don't occur such as heartbeat events we shouldn't see.
[YOCTO #11045]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The commands:
bitbake nodejs-native;
bitbake nodejs-native -c clean;
bitbake nodejs-native;
bitbake -c addto_recipe_sysroot nodejs-native;
bitbake -c devshell nodejs-native;
bitbake -c addto_recipe_sysroot nodejs-native;
bitbake -c devshell nodejs-native;
never result in npm in the sysroot within devshell. The reason is the
addto_recipe_sysroot stamp isn't removed when do_fetch is run but the sysroot
is cleaned.
With this patch, the second devshell will contain npm, which I think is probably
the best outcome we can hope for here.
[YOCTO #11461]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 037ed2eee097fb9d0f52756b7c16635d931867e8.
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit dc7da9756bcfc4b3c93e835c408b1cd0ecb4220f.
|
|
This reverts commit f140783a35cddc1beb025e57170a290be5ebb37e.
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 89c73937897fdb2a70e706c8affddf50308cebbc.
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you specify relative paths on the commandline, bad things can
happen as the directory relative to bblayers.conf may not be the same.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 42dfe27f3778b358337fff9ec56b8f569f789522.
Breaks qemuarm core-image-sato build in do_rootfs
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit a8e322a4c030f1d61e4f3049734932bb2e6f3887.
Breaks ESDK:
ERROR: OE-core's config sanity checker detected a potential misconfiguration.
Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
Following is the list of potential problems / advisories:
Your gcc version is older than 4.5 or is not working properly. Please verify you can build and link something that uses atomic operations, such as:
__sync_bool_compare_and_swap (&atomic, 2, 3);
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 8ea1108cad6f75b92dece1cb3095529ce8c0a3e9.
|
|
This is a WIP to add a make job server into bitbake. This means the pool
of make tokens is central and shared by all tasks executed instead of the
current one job pool per task. Currently we can end up with many more
make subprocesses executing that is intended or optimal.
Implementation wise, make usually uses a pipe for this functionality. Here
we substitute a named pipe (fifo) and intercept the make commands, passing
in file descriptors to the central fifo. This assumes knowledge of make's
internal API, on the plus side it hasn't changed since 1999.
Looking to the future we could dynamically control the pool but one step
at a time.
TODO:
* Remove hardcoded /tmp/makefifo and use something in TMPDIR or similar
(alongside the lock file?)
* Remove hardcoded make threads number and set from PARALLEL_MAKE
* If PARALLEL_MAKE = "", don't set MAKEARGS (currently parallelism is
set everywhere) (need to check for -j in make commandline)
I'm sending this out so at least the code is available to people. Its
not ready for merging in its current form but might be the basis for
someone else to finish this up, it also gives us something we can test
the performance implications with.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit a9f86cbdb2a0e4b848020df0c14d532238ca99f9.
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit f5c3b56e5e2cc32380895ca476ca7dbbdf86e81c.
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit e6003ca8829a389e1e5ae9e8410c025735087e77.
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie