aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2017-07-18recipetool: reimplement fetching with normal fetch/unpack taskspaule/fixes2dPaul Eggleton
Now that we have the ability to run the tasks in a more standard context through tinfoil, change recipetool's fetching code to use that to fetch files using it. This has the major advantage that any dependencies of do_fetch and do_unpack (e.g. for subversion or npm) will be handled automatically. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13knotty: make it possible to use termfilter without consolesPaul Eggleton
This isn't useful for knotty itself, but for use from tinfoil in case we can't get access to either the console or errconsole, allow either to be unspecified (None). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13fetch2: fire an event when there are missing checksumsPaul Eggleton
If BB_STRICT_CHECKSUMS is set to anything other than "1" i.e. we're not going to raise an error, then fire an event so that scripts can listen for it and get the checksums. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13fetch2: allow hiding checksum warningPaul Eggleton
If BB_STRICT_CHECKSUMS is set to "ignore" then don't display a warning if no checksums are specified in the recipe. This is not intended to be used from recipes - it is needed when we move to using more standard code paths to fetch new files from scripts i.e. where we don't know what the checksums are in advance. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13fetch2: fix indentationPaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13recipetool: ensure meaningful error for malformed tarballsPaul Eggleton
If you pointed recipetool at a URL that should be a tarball e.g. https://tls.mbed.org/download/start/mbedtls-2.4.2-apache.tgz but instead it returns an HTML page, we try to unpack it, gzip complains but the operation doesn't seem to fail - instead we just get back an empty source tree. Change the checks to account for this - if the source tree is empty, check if the downloaded file in DL_DIR looks like an HTML file and error accordingly if it is. If it's not, error out anyway because no source was unpacked and it should have been (otherwise we just blindly set up EXTERNALSRC for this which is pointless). Fixes an aspect of [YOCTO #11407]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13classes/staging: change fixme debug message from note to debugPaul Eggleton
These messages were added for debugging during the recipe specific sysroot work. They may still be useful but they don't need to be notes - if they are they show up in recipetool / devtool output when fetching source. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13tinfoil: add more doc commentsPaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13cooker: fix whitespacePaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13cache.py: fix whitespacePaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13tinfoil: add simple API for getting cached recipe informationPaul Eggleton
A common task for tinfoil-using scripts is to iterate over all recipes. This isn't too difficult with the current API, but the pkg_* variables are a little awkward and are really designed for bitbake's internal usage - and it gets a bit more difficult when you want to access some of the other information such as packages and rprovides. To resolve this, create a new recipe info class and add an all_recipes() function to generate this for all recipes. Also add a get_recipe_info() function to get the information for a specific recipe (by PN). (It might perhaps be suggested that we already have a structure similar to this in the cache, however the one we add here is designed for external use and allows the internal structures to change if needed without affecting the API). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13tinfoil: enable access to additional cached itemsPaul Eggleton
Add access to fn_provides, packages, packages_dynamic and rproviders on the recipecache object. This requires an additional corresponding command plumbing to be added. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13cookerdata: fix indentation and trailing spacesPaul Eggleton
Fix one incorrect indentation and several lines with trailing spaces. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13devtool: extract: refactor kern-tools-native handlingPaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13recipetool: refactor code for ensuring npm is availablePaul Eggleton
Across devtool and recipetool we had an ugly set of code for ensuring that we can call an npm binary, and much of that ugliness was a result of not being able to run build tasks when tinfoil was active - if recipetool found that npm was required and we didn't know beforehand (e.g. we're fetching from a plain git repository as opposed to an npm:// URL where it's obvious) then it had to exit and return a special result code, so that devtool knew it needed to build nodejs-native and then call recipetool again. Tinfoil now allows us to run build tasks directly, so we can take advantage of that instead and throw away the ugly code at the same time. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-13tinfoil: add functionality for running full buildsPaul Eggleton
Up to this point, if you wanted to run build tasks in the normal way they get run, there was no other way than to shell out to bitbake. As long as we're prepared to handle the events produced, we can create a wrapper around calling the buildTargets command. Borrow code from knotty to do this so that we get the expected running task display (courtesy of TermFilter) and Ctrl+C handling. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22lib/bb/ui/uihelper: indicate to caller of eventHandler() if events handledPaul Eggleton
It is useful for the caller to know whether the uihelper has handled the event passed so that it can skip other event handling code if so. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22knotty: fix indentationPaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22lib/bb/event: refactor printing eventsPaul Eggleton
We really ought to have just one place where the string representation of these events is produced. This doesn't take any real control away from the UI - if an alternative representation is desired, that can still be made. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22tinfoil: set a flag when recipes have been parsedPaul Eggleton
Make it easy to determine if recipes are parsed (and thus information about available recipes is in memory). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22tinfoil: add a parse_recipes() functionPaul Eggleton
Python style recommends underscore based naming rather than camelCase, and thus the former has been used for most of tinfoil's functions. Add an underscored version of parseRecipes() for consistency and change the one place we call it to use the new version. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22tinfoil: add internal mode to build_file() functionPaul Eggleton
In OE's devtool we want to repeatedly run build_file() without showing unnecessary messages and triggering buildhistory for each call. build_file() is just a wrapper around the buildFile command. Change the final "hidewarning" parameter of the buildFile command to "internal" and have this call a new buildFileInternal() function without triggering any of the normal build events, silencing the normal info messages from the runqueue ("Executing RunQueue Tasks", "Tasks Summary" etc.) and avoiding calling parseConfiguration() which we've already done at this point. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22classes/buildhistory: fix failures collecting output signaturesPaul Eggleton
It's possible for tasks to stage symlinks that point to non-existent files; an example is ncurses-native.do_populate_sysroot. There wasn't any error checking here so this broke the build when "task" was included in BUILDHISTORY_FEATURES. In any case we shouldn't be following symlinks and getting the sha256sum of the link target - we need concern ourselves only with the target path, so check if the file is a link and sha256 the target path instead if it is. If it's neither a regular file nor a symlink (perhaps a pipe or a device), just skip it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22oe-init-build-env-memres: swap parameter orderPaul Eggleton
This script expected the port number to be specified first, which is somewhat counter-intuitive especially if you're used to oe-init-build-env; besides, in local usage you are unlikely to need to specify a custom port. Given that few people are using this yet (based on the issues I have recently fixed), switch the arguments around so that the two scripts behave consistently. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-06-22bitbake: fetch: fix handling of files with incorrect checksums from a premirrorJoshua Lock
Ensure that when an item fetched from a premirror has an invalid checksum the fetcher falls back to the usual logic of trying the upstream and any configured mirrors. (Bitbake rev: 022adb30dbb0df764c9fb515918cb9a88e4f8d6f) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22bitbake: event: drop some unused eventsPaul Eggleton
These were all used by support code for the Hob UI which has been removed - nothing will currently fire or respond to these events. (Bitbake rev: 03ba8db0595723286ad462fa197f862e1efdf0de) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22bitbake: toaster: noweb should init databaseDavid Reyna
When the 'noweb' option for Toaster is used, perform the database check/create if the Toaster database does not yet exist. This will allow Toaster to not fail if the first use is with 'noweb'. This avoids potentially clashing database updates if there are multiple overlaping 'noweb' sessions (for example with a CI system). If the user wished to update the database, they can either use the explicit "lsupdate" command or (re)start a web hosted Toaster session (which is gated by the webserver's PID). [YOCTO #11378] (Bitbake rev: 910b96b9894c712aa32b5d4dadda88b766d86e35) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22bitbake: toaster: get_last_build_id not called correctlyDavid Reyna
The method "get_last_build_id" missing the "()" in several calls. This has been failing silently with Django 1.8 but intermittently crash with Django 1.9. [YOCTO #11570] (Bitbake rev: c29e137a6e65317b8a36804126511c2137878715) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22bitbake: toaster: add getMessage to MockEventDavid Reyna
The MockEvent needs to not only stand in for Toaster and Bitbake quick events, it also needs to stand in for LogRecord, and for that it needs to provide the new getMessage method. [YOCTO #11440] (Bitbake rev: d1ac359d460b1abe9815f323b3fd4cd0231cde6c) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22bitbake: toaster: fail on layers with sub-layerDavid Reyna
For layers directories with sub-layers (for example meta-intel), Toaster breaks when trying to exact match a dependency event for that sub-layer against the top level layer directory paths. Given that top the level layer directory paths are unique, adding a test to see if the dependency path is a subset of a top level path is also unique. Also, since a warning was issued the processing should not fail on the assert but instead should continue and gather the recipes and tasks. [YOCTO #11149] (Bitbake rev: 6f1a42af19220a6a8d75790893da995e84ecd7aa) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22bitbake: toaster: add ID's to build menu linksDavid Reyna
Add IDs to the build page menu lines, for example "Tasks", "Recipes", "Time", "CPU Time", "Disk I/O", and so forth. This will support test automation. [YOCTO #11337] (Bitbake rev: 49d11490fad683a96f71de21f55035398ad8247a) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22bitbake: toaster: add ID's to navigation linksDavid Reyna
Add IDs to the navigation bar links "All builds", "All projects", and "Documentation. This will support test automation. [YOCTO #11335] (Bitbake rev: 5cbf6f67994fc42dae72ece191e5c9630fc4b5af) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22meta-yocto-bsp: bump to the latest linux stable kernel for the non-x86 BSPsKevin Hao
Bump to the latest stable kernel for 4.4, 4,9 and 4.10. (From meta-yocto rev: 294a7d2e1b8d7c54f6ec11804254e7028390c028) Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Updates to the section on submitting changes to YPScott Rifenbark
I added detail to the procedure about pushing a change to a contrib directory and then requesting a pull using the scripts. More detailed with examples. Also, clarified some wording in the section about submitting a patch through email. (From yocto-docs rev: 76a358ba588b0f36d14aef0a68efe30c4e16746a) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Updated checkout by branchKristi Rifenbark
Example was incorrect. Fixed it. (From yocto-docs rev: 7ae1c636e7a8c0448515f339476327bcaf21e63c) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22ref-manual: uClibc Replaced by musl from Yocto 2.2Changhyeok Bae
(From yocto-docs rev: ed7e62f5937576d7878eb955fda12c18e787db18) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22ref-manual: Fixed spelling error "containe"Kristi Rifenbark
(From yocto-docs rev: cf8032bf7c3bc484b63ee46c195ee1537b1f817f) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual, ref-manual: Re-wrote the introsScott Rifenbark
Fixes [YOCTO #11630] I updated both the introductory sections of the ref-manual and the dev-manual. These need to work more tightly with the projected YP documentation set. (From yocto-docs rev: 6a28537d03fad8a53198edc7f9a6229f4e58e551) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Updated first section title to "Welcome"Scott Rifenbark
I want this first section to be titled "Welcome" rahter than "Introduction." It is more in line with the YP QS and the YP ref manual. (From yocto-docs rev: 113b82c589c93a33a2a956790668e8c0a28894ec) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22yocto-project-qs: Updated "Welcome" section.Scott Rifenbark
I moved the bit about using a VM if you don't have a Linux machine to a note as it broke the natural flow of this section. Also, inserted a link to the comprehensive list of links and other documentation references to the existing list of other information. (From yocto-docs rev: 73750063580ef7bc20bd445adc3e2b0b01e2257f) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22yocto-project-qs: Updated phonetic rendition of "poky"Scott Rifenbark
I changed from "Pah-key" to "Pah-kee", which is more universal and also matched what is used in the dev-manual terms section. (From yocto-docs rev: 843e832f2416b0e3b0e9d941bf7755f206517603) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22documentation: Changes section head for sumbitting change to YPScott Rifenbark
Fixes [YOCTO #11630] I changed the section heading for the section that describes how to make a change to the YP. It was passive. I need it to be active and in line with the "how-to" dev-manual. (From yocto-docs rev: 340e1b6ab9806ec11ec894d4472bc2c937c2b058) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Updated the section on the YP development environmentScott Rifenbark
Fixes [YOCTO #11630] Extensively updated the development environment section by creating a task-oriented section. (From yocto-docs rev: d440f0b5ceb1aca57a73d8bced515ae0bd3188f7) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Updated "How to submit a change" section.Scott Rifenbark
Fixes [YOCTO #11630] The section on how to submit a change was pretty much a procedure section. I did some rewriting to make it more that way. (From yocto-docs rev: d7edce9268ee5cae96c09c79fe34d5d2dbb701e0) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Fixed a DocBook issue with formatting "bug-id"Scott Rifenbark
I discovered a mis-placed block of DocBook text that was causing the string "bug-id" to appear in a random spot in the "Patch Submission Details" section. Re-ording this block fixed the problem. (From yocto-docs rev: 6cf0e8ff0d2b176c1faf492c7bdd7d38e23e4c98) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Updated syntax for oe-run-native scriptKristi Rifenbark
OE native script now requires a native tool as part of the command line. I added that to the example. Removed a bad help command at the end of the section. (From yocto-docs rev: 5eb965ea51d669ee3bac01050b42bee984572199) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Fixed two grammar errorsKristi Rifenbark
(From yocto-docs rev: 4e68646edf078c23e414a300b37cb30774b28be7) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Updated "Kernel Overview" sectionKristi Rifenbark
The list of supported kernels was out of date. Edited with updated information. (From yocto-docs rev: b287f76905637834ec6342285b99abd2c5594597) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22ref-manual: Added "see also" informational note at beginning.Scott Rifenbark
(From yocto-docs rev: 3af3d821b6e26fd352223f3f4196e0e200614203) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22dev-manual: Created "Working With Git Repositories" sectionScott Rifenbark
Fixes [YOCTO #11630] This new section has three new procedures for now: 1) cloning the poky repository, 2) checking out a branch based on an upstream repository branch name, and 3) checking out a branch based on an upstream tag name. More to be added for other key repositories. (From yocto-docs rev: 4cbcd952ba395b19be5374470a95630a4bc6ffa7) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>