aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2017-07-21poky: Remove machine specific kernel overridesbavery/mv_linux-yocto_into_machineconfbrian avery
This patch removes the machine specific overrides in poky.conf for all the qemu machines. With this patch, if you set PREFERRED_VERSION_linux-yocto="X%" then all the qemu machines as well as the sample hardware boards will use that version. Without this patch, the hardware boards would have used version X but the qemu machines would have needed to have a PREFERRED_VERSION_linux-yocto_qemuarm="X%" for qemuarm to use that version, for example. If we need to force a specific machine (hardware or qemu) to a particular default version we should add it to the machine conf files for that machine, not in the distro conf. Signed-off-by: brian avery <brian.avery@intel.com>
2017-07-21meta-yocto-bsp: Remove superfluous kernel setbrian avery
The genericx86 and genericx86-64 machine confs do not need to set the PREFERRED_PROVIDER_virtual/kernel as they will use the one set by the distribution. If there needed to be a different version for one of these machines in the future, we could add it in to the machine.conf. Signed-off-by: brian avery <brian.avery@intel.com>
2017-07-21bison: reduce local pending patchesDengke Du
(From OE-Core rev: 119dd0889f34d88a5261383d743709db32bf3a1b) Signed-off-by: Dengke Du <dengke.du@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bluez5: update patch statusMaxin B. John
Update the status of following patch from Pending to Accepted: a) 0001-hciattach-bcm43xx-fix-the-delay-timer-for-firmware-d.patch (From OE-Core rev: f9ccac5dd359e9b874a39cd879ea23c841085eab) Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: fetch2/wget.py: improve error handling during sstate checkPatrick Ohly
When the sstate is accessed via HTTP, the existence check can fail due to network issues, in which case bitbake silently continues without sstate. One such network issue is an HTTP server like Python's own SimpleHTTP which closes the TCP connection despite an explicit "Keep-Alive" in the HTTP request header. The server does that without a "close" in the HTTP response header, so the socket remains in the connection cache, leading to "urlopen failed: <urlopen error [Errno 9] Bad file descriptor>" (only visible in "bitbake -D -D" output) when trying to use the cached connection again. The connection might also get closed for other reasons (proxy, timeouts, etc.), so this is something that the client should be able to handle. This is achieved by checking for the error, removing the bad connection, and letting the check_status() method try again with a new connection. It is necessary to let the second attempt fail permanently, because bad proxy setups have been observed to also lead to such broken connections. In that case, we need to abort for real after trying twice, otherwise a build would just hang forever. [YOCTO #11782] (Bitbake rev: 6fa07752bbd3ac345cd8617da49a70e0b2dd565f) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: toaster: trim build target inputDavid Reyna
Trim the entered built target value so that Toaster is not confused with no real targets nor a ghost second target. [YOCTO #11727] (Bitbake rev: dd83c732b945ba6f9b7cdf66f6e88cfd15a745a0) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: toaster: set clone progress default to offDavid Reyna
Set the clone progress to be off by default for the benefit of command line projects and 'Local Yocto' builds. For Toaster managed projects that do use the clone feature the clone progress status is already explicitly set by the existing code and thus displayed. [YOCTO #11744] (Bitbake rev: d25a3088150b2a277242027c12becdeb88c45517) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: toaster: include setscene in task progressDavid Reyna
Change the task progress counting from the task order field which excludes setscene to the task completion field which counts all completed tasks regardless of type. [YOCTO #9971] (Bitbake rev: 485b328430dcdfd0b52f09f01ebd490d09459ea3) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: Update to version 1.35.0 (development version with server rework ↵Richard Purdie
changes) (Bitbake rev: eef7a1a3eb0365da5ed2bc9688203fba3b6a61b5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21subversion: Upgrade 1.9.5-> 1.9.6Richard Purdie
(From OE-Core rev: 5212d88104b7a53d4bd8bf2320aca9455099ac80) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21argparse_oe: Add int_positive typeAníbal Limón
Sometimes only expect positive values from cmdline so it's better to filter at parsing cmdline step instead of validate later. (From OE-Core rev: 3ef5b518febd047bf90a0955fa2b9fb78ba6dde5) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21oe/copy_buildsystem: check_sstate_task_list also pop BBPATH from envAníbal Limón
The BBPATH environment could be set and can make a failure when try to build an extensible sdk because it will look the bitbake.lock file in the original build folder. Example: $ export BBPATH=`pwd` $ bitbake core-image-minimal -c populate_sdk_ext ERROR: bitbake failed: ERROR: Only one copy of bitbake should be run against a build directory ERROR: core-image-minimal-1.0-r0 do_populate_sdk_ext: Function failed: copy_buildsystem (From OE-Core rev: 33634b4c38d84e1c5d06056766933f1fe4f47e8d) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21oeqa/selftest/{context,case}: Handle KeyboardInterrupt/SIGINT and SIGTERMAníbal Limón
In order to avoid corrupt local.conf and bblayers.conf adds signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block to restore previously backuped configuration. [YOCTO #11650] (From OE-Core rev: 9419c81e69d2facc82e39c846466670c09e6b444) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21selftest/cases/package: Call parent setUpClass methodAníbal Limón
Since config paths are now passed in Test context the setUpClass method is expected to be call. (From OE-Core rev: 7281c995ff2b009c3fb23c7af1d91fe106ca8f87) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21oeqa/core/loader: Generate function _make_failed_test dynamicallyAníbal Limón
Python versions has different features from branches 3.4.x, 3.5.x and 3.6.x, i expected in wrong mode that was incremental for example changes in 3.4.4 be in 3.5.x but that's not true. The _make_failed_test internal method differs and is only available in certain versions >= 3.4.4 and in 3.5.x and 3.6.x branches but not realeses have been made including it. So to avoid futher problems inspect the _make_failed_test and generates function definition according what parameters are needed, the unique supossition is that exception argument is always passed. Related to, http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=d8380d098a290510b442a7abd2dd5a50cabf5844 (From OE-Core rev: b4740d2b325a80bcecc5e56dff9add9081fcd31b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21oeqa/eSDK: Ignore errors during directory cleanupRichard Purdie
The cleanup can fail with: ERROR [0.000s]: tearDownClass (eSDK.oeSDKExtSelfTest) FileNotFoundError: [Errno 2] No such file or directory: 'bitbake.sock' which is due to bitbake taking a small amount of time to shut down the server. The easiest fix is just to ignore these kinds of errors, bitbake shouldn't create any new files during shutdown. (From OE-Core rev: 68009f4eb323e07cec8c4421d546bd3125dec033) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21recipetool: Setup logging before calling prepareRichard Purdie
This avoids test failures like: ====================================================================== ERROR [0.946s]: test_recipetool_load_plugin (recipetool.RecipetoolTests) INFO - ---------------------------------------------------------------------- INFO - Traceback (most recent call last): File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f return func(*args, **kwargs) File "/media/build1/poky/meta/lib/oeqa/selftest/cases/recipetool.py", line 514, in test_recipetool_load_plugin with open(srcfile) as fh: FileNotFoundError: [Errno 2] No such file or directory: 'NOTE: Starting bitbake server...\n/media/build1/poky/meta-selftest/lib/recipetool/bbpath.py' INFO - ---------------------------------------------------------------------- which comes from: $ recipetool --quiet pluginfile NOTE: Starting bitbake server... /media/build1/poky/meta-selftest/lib/recipetool/bbpath.py since there is corruption in the output despite the --quiet option. Setting the logging up before calling tinfoil.prepare() resolves this. (From OE-Core rev: e124eb73e6a66abbf43dffd82b834e906163614b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21devtool: Setup logging before calling prepareRichard Purdie
This avoids test failures like: INFO - ====================================================================== INFO - FAIL [1.755s]: test_devtool_layer_plugins (devtool.DevtoolTests) INFO - ---------------------------------------------------------------------- INFO - Traceback (most recent call last): File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f return func(*args, **kwargs) File "/media/build1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1354, in test_devtool_layer_plugins self.assertEqual(result.output, s[::-1]) AssertionError: "NOTE: Starting bitbake server...\noY senu[36 chars]rciM" != "oY senuZ s'enoynA morF tiforP oN edaM tfosorciM" - NOTE: Starting bitbake server... oY senuZ s'enoynA morF tiforP oN edaM tfosorciM INFO - ---------------------------------------------------------------------- since there is corruption in the output. Setting the logging up before calling tinfoil.prepare() resolves this. (From OE-Core rev: 3c479fb17ae4d3e7e5f0889af0f68257ef66475c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21oe-pkgdata-util: Setup logging before calling prepareRichard Purdie
This avoids test failures like: INFO - ====================================================================== INFO - FAIL [1.046s]: test_find_path (pkgdata.OePkgdataUtilTests) INFO - ---------------------------------------------------------------------- INFO - Traceback (most recent call last): File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f return func(*args, **kwargs) File "/media/build1/poky/meta/lib/oeqa/selftest/cases/pkgdata.py", line 50, in test_find_path self.assertEqual(result.output, 'zlib: /lib/libz.so.1') AssertionError: 'NOTE: Starting bitbake server...\nzlib: /lib/libz.so.1' != 'zlib: /lib/libz.so.1' - NOTE: Starting bitbake server... zlib: /lib/libz.so.1 INFO - ---------------------------------------------------------------------- since there is corruption in the output. Setting the logging up before calling tinfoil.prepare() resolves this. (From OE-Core rev: 39c1bf1feccc8dca57160a1e3cd8a50e8c30ef02) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21recipetool: create: refactor code for ensuring npm is availablePaul Eggleton
Across devtool and recipetool we had an ugly set of code for ensuring that we can call an npm binary, and much of that ugliness was a result of not being able to run build tasks when tinfoil was active - if recipetool found that npm was required and we didn't know beforehand (e.g. we're fetching from a plain git repository as opposed to an npm:// URL where it's obvious) then it had to exit and return a special result code, so that devtool knew it needed to build nodejs-native and then call recipetool again. Now that we are using real build tasks to fetch and unpack, we can drop most of this and move the code to the one place where it's still needed (i.e. create_npm where we potentially have to deal with node.js code in a plain source repository). (From OE-Core rev: 8450de16ddb02d863204b411a94c6d84e0f88817) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21classes/base: set do_unpack dependencies for rpm filesPaul Eggleton
If we're unpacking any rpm (not just .src.rpm) then we'll need to call rpm2cpio.sh. As of OE-Core rev a7da1aade118d1ccf1b286f82556cd9f706bd2a4 that script no longer uses file-native, but there is a chance that the rpm file will be compressed internally using xz (for example, rpms from Fedora) and therefore the script will need xzcat provided xz-native. (From OE-Core rev: 2f31f1795bc0c85b1646bc7d9596bbe778cb84e5) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21recipetool: create: reimplement fetching with normal fetch/unpack tasksPaul Eggleton
Now that we have the ability to run the tasks in a more standard context through tinfoil, change recipetool's fetching code to use that to fetch files using it. This has the major advantage that any dependencies of do_fetch and do_unpack (e.g. for subversion or npm) will be handled automatically. This also has the beneficial side-effect of fixing a recent regression that prevented this fetch operation from working with memory resident bitbake. Also fix devtool's usage of fetch_uri() at the same time so that we can completely replace it. Fixes [YOCTO #11710]. (From OE-Core rev: 9a47a6690052ef943c0d4760630ee630fb012153) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21recipetool: create: eliminate second fetch for packagesPaul Eggleton
When dealing with package files (.rpm, .ipk etc.) we need to unpack them ourselves to get the metadata, which is thrown away when the fetcher unpacks them. However, since we've already fetched the file once, I'm not sure as to why I thought I needed to fetch it again - we can just get the local path and then unpack it directly. (From OE-Core rev: be45e9b17e9dbc8c2594d3a939be377ab0720a7c) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21recipetool: create: ensure meaningful error for malformed tarballsPaul Eggleton
If you pointed recipetool at a URL that should be a tarball e.g. https://tls.mbed.org/download/start/mbedtls-2.4.2-apache.tgz but instead it returns an HTML page, we try to unpack it, gzip complains but the operation doesn't seem to fail - instead we just get back an empty source tree. Change the checks to account for this - if the source tree is empty, check if the downloaded file in DL_DIR looks like an HTML file and error accordingly if it is. If it's not, error out anyway because no source was unpacked and it should have been (otherwise we just blindly set up EXTERNALSRC for this which is pointless). Fixes an aspect of [YOCTO #11407]. (From OE-Core rev: 8496113b63d5a5d1f99056610c0fdb972a6200d4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21devtool: extract: refactor kern-tools-native handlingPaul Eggleton
When extracting linux-yocto kernel source, we don't need to dance around shutting down and starting up tinfoil anymore, we can just execute the tasks as needed when needed using tinfoil's new build_targets() function. This allows us to tidy up the code structure a bit. (From OE-Core rev: 5c7f5031023fb74b5f2f26d6b3c829981f2f54d2) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21classes/staging: change fixme debug message from note to debugPaul Eggleton
These messages were added for debugging during the recipe specific sysroot work. They may still be useful but they don't need to be notes - if they are they show up in recipetool / devtool output when fetching source. (From OE-Core rev: a0e93d5c5dcf59d1898a3db727a5ab2d75e3d20e) (From OE-Core rev: bc1a4af950a2600028d89b7dcb4ff8f409b52739) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21oe-init-build-env-memres: Drop itRichard Purdie
With the new server structure we no longer need this separate environment init script. Just set BB_SERVER_TIMEOUT to be greater than zero and bitbake will remain in memory and the UI will auto-reconnect to it. Also clean out the old shutdown code from oe-init-build-env which also doesn't make sense now. (From OE-Core rev: a17724d687f9c04dc21f2f369da2b492f7119ba0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21conf/sanity: Update minimum bitbake verison to 1.35.0 for server reworkRichard Purdie
(From OE-Core rev: b63fcffe8b6a3f94d4812cd03fb974e173eff3e5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: fetch2: fire an event when there are missing checksumsPaul Eggleton
If BB_STRICT_CHECKSUMS is set to anything other than "1" i.e. we're not going to raise an error, then fire an event so that scripts can listen for it and get the checksums. (Bitbake rev: 8b2ccb4b865f2df118ef668847df682a83f9c500) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: fetch2: allow hiding checksum warningPaul Eggleton
If BB_STRICT_CHECKSUMS is set to "ignore" then don't display a warning if no checksums are specified in the recipe. This is not intended to be used from recipes - it is needed when we move to using more standard code paths to fetch new files from scripts i.e. where we don't know what the checksums are in advance. (Bitbake rev: f15ca7339de8a448a93a14cf6130b3925178a920) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: tinfoil: add more doc commentsPaul Eggleton
We want this API to be easier to use, so add missing function documentation to help with that. (Bitbake rev: 3e0e002d6497caa987f327cd83ad4db82cca6810) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: tinfoil: add simple API for getting cached recipe informationPaul Eggleton
A common task for tinfoil-using scripts is to iterate over all recipes. This isn't too difficult with the current API, but the pkg_* variables are a little awkward and are really designed for bitbake's internal usage - and it gets a bit more difficult when you want to access some of the other information such as packages and rprovides. To resolve this, create a new recipe info class and add an all_recipes() function to generate this for all recipes. Also add a get_recipe_info() function to get the information for a specific recipe (by PN). (It might perhaps be suggested that we already have a structure similar to this in the cache, however the one we add here is designed for external use and allows the internal structures to change if needed without affecting the API). (Bitbake rev: 308994028e59735ca726c5d2c1f0f85baccfe89d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: tinfoil: enable access to additional cached itemsPaul Eggleton
Add access to fn_provides, packages, packages_dynamic and rproviders on the recipecache object. This requires an additional corresponding command plumbing to be added. (Bitbake rev: 3df9b7c615174a6557581f3cd157842a28f6bb26) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: tinfoil: add functionality for running full buildsPaul Eggleton
Up to this point, if you wanted to run build tasks in the normal way they get run from a python script, there was no other way than to shell out to bitbake. Worse than that, you couldn't have tinfoil active during that because only one bitbake instance could be running at once. As long as we're prepared to handle the events produced, we can create a wrapper around calling the buildTargets command. Borrow code from knotty to do this in such a way that we get the expected running task display (courtesy of TermFilter) and Ctrl+C handling. (Bitbake rev: 43761eee756be52a1021be53a40dc591a6c35fa7) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: knotty: make it possible to use termfilter without either consolePaul Eggleton
This isn't useful for knotty itself, but for use from tinfoil in case we can't get access to either the console or errconsole, allow either to be unspecified (None). (Bitbake rev: 7544de437fc66b81502ecdb5db859182c45827cb) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: lib/bb/ui/uihelper: indicate to caller of eventHandler() if events ↵Paul Eggleton
handled It is useful for the caller to know whether the uihelper has handled the event passed so that it can skip other event handling code if so. (Bitbake rev: be498abfbbb19bdd31e5b53713a74049007e3737) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: lib/bb/event: refactor printing eventsPaul Eggleton
We really ought to have just one place where the string representation of these events is produced. This doesn't take any real control away from the UI - if an alternative representation is desired, that can still be made. (Bitbake rev: cb15db2a799be6d8eab9a2a43a9a573f89229cff) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: tinfoil: set a flag when recipes have been parsedPaul Eggleton
Make it easy to determine if recipes are parsed (and thus information about available recipes is in memory). (Bitbake rev: 7efde2df2ff25063d36ac015146f1975284a69ff) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: tinfoil: add a parse_recipes() functionPaul Eggleton
Python style recommends underscore based naming rather than camelCase, and thus the former has been used for most of tinfoil's functions. Add an underscored version of parseRecipes() for consistency and change the one place we call it to use the new version. (Bitbake rev: 821f6c41d850752d2bcc2ccd4f8e75b2897a0a3e) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: tinfoil: add internal mode to build_file() functionPaul Eggleton
In OE's devtool we want to repeatedly run build_file() without showing unnecessary messages and triggering buildhistory for each call. build_file() is just a wrapper around the buildFile command. Change the final "hidewarning" parameter of the buildFile command to "internal" and have this call a new buildFileInternal() function without triggering any of the normal build events, silencing the normal info messages from the runqueue ("Executing RunQueue Tasks", "Tasks Summary" etc.) and avoiding calling parseConfiguration() which we've already done at this point. (Bitbake rev: ba53e067a2d448dd63b4ca252557ce98aa8e6321) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: bb/main.py: fix logic for --observe-onlyRobert Yang
Fixed: $ bitbake --observe-only FATAL: '--observe-only' can only be used by UI clients connecting to a server. And even: $bitbake --observe-only -B localhost:-1 FATAL: '--observe-only' can only be used by UI clients connecting to a server. This was beucase the brackets were in a wrong position. (Bitbake rev: 4ffc91a2b3eb13e98078e6b1913f056a0c1797bc) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: bb/main.py: avoid starting server when not neededRobert Yang
Fixed 1: $ . ../poky/oe-init-build-env-memres . $ bitbake -m $ bitbake -m # shutdown it again $ bitbake -m Starting bitbake server... Terminated bitbake server. It starts the server and kill it which is redundant. $ bitbake -m $ bitbake --status-only Starting bitbake server... It starts the server which is not what we need. (Bitbake rev: 335fd0106359dddfe1eeb0c7d0bc03f8454b895c) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: bb/main.py: fix infinite loop for --server-onlyRobert Yang
Fixed: $ bitbake --server-only -B localhost:-1 Infinite loop Reconnecting to bitbake server... [snip] Break the loop when server-only can fix the problem. (Bitbake rev: c4d5c3dab5b35f3c6713efad1c8a23229ce4e56b) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: server/process.py: fix self.bitbake_lock.write()Robert Yang
There is no global var "configuration", so the old code hang at self.bitbake_lock.write(), and nothing wrote to bitbake.lock. I didn't figure out why it hang (but not print errors). Reproducer: $ bitbake -B localhost:-1 world -k Check bitbake.log, there was nothing, now fixed. (Bitbake rev: dbdd9010663cd8dcb328e1b6fb40e3a777293bc5) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: cookerdata/cooker: Restore original datastore upon client disconnectRichard Purdie
(Bitbake rev: 67ae612ab890965357d24a9bd35610cf813b79ba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: server: Rework the server API so process and xmlrpc servers coexistRichard Purdie
This changes the way bitbake server works quite radically. Now, the server is always a process based server with the option of starting an XMLRPC listener on a specific inferface/port. Behind the scenes this is done with a "bitbake.sock" file alongside the bitbake.lock file. If we can obtain the lock, we know we need to start a server. The server always listens on the socket and UIs can then connect to this. UIs connect by sending a set of three file descriptors over the domain socket, one for sending commands, one for receiving command results and the other for receiving events. These changes meant we can throw away all the horrid server abstraction code, the plugable transport option to bitbake and the code becomes much more readable and debuggable. It also likely removes a ton of ways you could hang the UI/cooker in weird ways due to all the race conditions that existed with previous processes. Changes: * The foreground option for bitbake-server was dropped. Just tail the log if you really want this, the codepaths were complicated enough without adding one for this. * BBSERVER="autodetect" was dropped. The server will autostart and autoconnect in process mode. You have to specify an xmlrpc server address since that can't be autodetected. I can't see a use case for autodetect now. * The transport/servetype option to bitbake was dropped. * A BB_SERVER_TIMEOUT variable is added which allows the server to stay resident for a period of time after the last client disconnects before unloading. This is used if the -T/--idle-timeout option is not passed to bitbake. This change is invasive and may well introduce new issues however I believe the codebase is in a much better position for further development and debugging. (Bitbake rev: 72a3dbe13a23588e24c0baca6d58c35cdeba3f63) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: daemonize: Various fixesRichard Purdie
Currently if this code is used with something like oeqa's xml logging it fails as sys.stdout is an io stream. Add in try/except to handle this case. Add a waitpid() call to remove a zombie whilst forking. Also, append to the logfile, don't overwrite it (otherwise debugging can be a real pain when the server is restarting for unknown reasons). (Bitbake rev: 4e780fa19fc720948cf73133f56f7d837ad9283a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: cookerdata: Add a function to find TOPDIRRichard Purdie
Finding the top level build directory is currently hard and relies on having a complete cooker being setup. Add a helper function which does the same thing without all the extra overhead. This is needed to be able to locate the bitbake lockfile and hence the socket for connecting clients in the new server model. (Bitbake rev: d196afe68032898c31a8599ca7d3ceba58d96b0a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: server: Remove base classes and inline codeRichard Purdie
In preparation for rewriting this code, expand the relatively useless base classes into the code itself. (Bitbake rev: a1c6151420d86bac658c08ae714647062edd6ef2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: event/command: Allow UI to request the UI eventhander IDRichard Purdie
The UI may want to change its event mask however to do this, it needs the event handler's ID. Tweak the code to allow this to be stored and add a command to query it. Use the new command in the process server backend. (Bitbake rev: f8cf2cb58b80ce74f756a11a9773b6b0e78d51ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>