summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2016-08-02bitbake-user-manual: Updated the variable expansion section.Scott Rifenbark
Fixes [YOCTO #9984] Added more detail to the examples that show the effects of variable expanison. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-08-02Revert "bitbake-user-manual: Added new section on command-line execution"Richard Purdie
This reverts commit 6f6cd0674fd1595f4e74b7da692e0c348b2660c6 as it was a duplicated commit.
2016-08-01lib/toaster: Fix missing new files from previous commitsRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-29toaster-tests: fix URL given for Chromedriver downloadElliot Smith
The link to the Chromedriver downloads page is dead, so put in the correct URL. Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-29toaster-tests: define capabilities for latest Firefox driverElliot Smith
For the latest Firefox versions, WebDriver requires a download of a separate binary and an additional capability to be defined on it. Modify our tests so that when "marionette" is set as the browser, this capability is defined on the Firefox driver. Also add a note to the README about the additional installation steps required. Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-29lib/bb/checksum: avoid exception on broken symlinksPaul Eggleton
If using OE's externalsrc with a source tree that is not tracked by git and contains broken symlinks, you can receive "TypeError: unorderable types: NoneType() < str()" within the file checksum code due to: checksums.sort(key=operator.itemgetter(1)) Don't add files with no checksum to the checksums list in order to avoid this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-29lib/bb/utils: show subprocess output in stack tracesRoss Burton
If better_exec() throws a subprocess.CalledProcessError then show the output to the user as it likely contains useful information for solving the problem. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-29fetch2: fix pickle issues while switching from master to krogothMaxin B. John
While switching from master to krogoth build with a common download directory, got a large number of warnings like the one listed below: WARNING: freetype-2.6.3-r0 do_fetch: Couldn't load checksums from donestamp /home/maxin/downloads/freetype-2.6.3.tar.bz2.done: ValueError (msg: unsupported pickle protocol: 4) These warnings are caused by the difference in pickle module implementation in python3(master) and python2(krogoth). Python2 supports 3 different protocols (0, 1, 2) and pickle.HIGHEST_PROTOCOL is 2 where as Python3 supports 5 different protocols (0, 1, 2, 3, 4) and pickle.HIGHEST_PROTOCOL is obviously 4. My suggestion is to use 2 since it is backward compatible with python2 (all the supported distros for krogoth provides python2 which supports pickle protocol version 2) Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26lib/bb/build: handle incomplete message fragments in log FIFORoss Burton
It's possible that the logging FIFO doesn't do a complete read (or the sender a complete write) with the result that an incomplete message is read in bitbake. This used to result in silently truncated lines but since 42d727 now also results in a warning as the start of the rest of the message isn't a valid logging command. Solve this by storing incoming bytes in a bytearray() across reads, and parsing complete messages from that. [ YOCTO #9999 ] Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster-tests: add tasks and recipes sub-page testsElliot Smith
Add tests for the tasks and recipes sub-pages of the build dashboard. [YOCTO #9833] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: set non-hideable columns for built recipes tableElliot Smith
None of the columns in the built recipes table are marked as not hideable, so it is possible to remove all the columns and make the table disappear. Set the recipe name and version columns as not hideable. Also rename the "Name" column to "Recipe", for consistency with the design and with other recipe tables. [YOCTO #9833] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: set non-hideable columns for build tasks tableElliot Smith
The task, recipe and order columns in the build tasks table should not be hideable. If they are, it's possible for the table to have all of its columns hidden so that it no longer displays. Set the hideable property to prevent these columns from being hidden. [YOCTO #9833] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: fix download URL for task logsElliot Smith
The task display template formatting had split the Django url template tag across two lines and broken it. This resulted in a gibberish URL for task logs. Fix by placing the tag and its arguments on a single line. [YOCTO #9837] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25fetcher2/__init__: Print command in case of ExecutionError in runfetchcmdMario Domenech Goulart
Signed-off-by: Mario Domenech Goulart <mario.goulart@bmw-carit.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: loadconf Partially add back some of the layerSource parsingMichael Wood
Partially add back a revised version of the layersource handling so that we can continue to support the old toasterconf.json and it's setup of the local project. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: admin Add Layer_Version to the admin-able modelsMichael Wood
If the migration didn't get the release conversion right for say, a local or imported layer it would be handy to be able to edit this in the django admin page. Also useful for developers to be able to tweak layers on the fly. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: orm Remove the layerindex specific up_branch fieldsMichael Wood
We don't need to keep track of layerindex data in our database. And using branch==release is very confusing in the schema. Instead use the existing Release definition to keep track of which release a layer_version is for. Remove the Branch model and all references to it. Create a migration path to convert from up_branches to their corresponding releases. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: lsupdates Add progress information and clean up loggingMichael Wood
Adds basic progress % information and provides better description of what is happening. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: tests: Remove references to LayerSource modelMichael Wood
Remove and replace layersource model references in the tests and test data. Remove the orm/test as this only tested LayerSource interactions which have now been removed. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: Replace references to LayerSource modelsMichael Wood
Replace references to the now deprecated layersource models across Toaster with the new enums for layer source types. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: lsupdates Add layerindex fetcherMichael Wood
Move and refactor the layerindex layer source update mechanism so that we don't have to track the layerindex objects in the toaster database. Move this out of the orm and into the management command. Paves the way for future improvement to allow you to specify a layer index server as an argument to the command. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: models Remove LayerSource models and replace with enumMichael Wood
We had a lot of complexity around different layer sources to allow for multiple sources and different priorities for these source. This was implemented using rigged abstract classes which represented the different layer sources when in fact just an enum/flag on the layer_version object is sufficient for our needs. Remove the LayerSourcePriority object as this is not needed. We no longer have a problem of multiple layers coming from multiple sources so this is not needed. Two migrations are added to first remove the child models which represented layersources. Then a second migration is needed to remove the LayerSource model it's self as Django can't understand the non-standard base class dependency. Triggering this issue: https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies Clean up a number of flake8 warnings in classes which were modified. [YOCTO #9853] Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: bldcollector admin Remove LayerSourceAdminMichael Wood
Remove the LayerSource admin from django admin interface. LayerSources are not going to be manageable from the admin interface. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25toaster: loadconf remove Loading LayerSourcesMichael Wood
We don't need to configure layer sources in the initial configuration as this information is provided by the models. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25lib/bb/progress: avoid possibility of start event being reported twicePaul Eggleton
In MultiStageProgressReporter, set a guard when we start the progress so that it can't happen more than once. This fixes "Initialising tasks.." being shown twice in succession when running bitbake in non-interactive terminal mode. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25knotty: don't display ETA for tasks with progressPaul Eggleton
It turns out that progress information we can extract from a task is rarely apportioned closely enough to the time taken for the ETA to be accurate, so showing it is going to be misleading most of the time for anything but the most basic of examples. Let's just remove it and avoid misleading (or worse, annoying) the user. Fixes [YOCTO #9986]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25knotty: fix some minor bugs in BBProgressPaul Eggleton
If you specify custom widgets then we don't want to assume where the "extra" position is - you should have to specify it, and if it isn't specified it shouldn't just wipe out the last widget or you can start to see odd behaviour if you're modifying the code. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-22fetch: copy files with -HEnrico Scholz
When using a PREMIRROR with plain (non-unpack) files, a SRC_URI like SRC_URI = "file://devmem2.c" will cause devmem2.c to be a symlink in the WORKDIR pointing to the local PREMIRROR. Trying to apply a patch on this file will either modify the file on the PREMIRROR or will fail due to sanity checks: ERROR: devmem2-1.0-r7 do_patch: Command Error: 'quilt --quiltrc /cache/build-ubuntu/sysroots/x86_64-oe-linux/etc/quiltrc push' exited with 1 Output: Applying patch devmem2-fixups-2.patch File devmem2.c is not a regular file -- refusing to patch Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-22cache: Don't interleave pickle cache file writingRichard Purdie
For some reason the data written in this way is coming back out the files out of order. I've not been able to simplify the test case to a point where this was standalone reproducible. Simplify the code and write out the cache files sequentially since this seems to avoid the errors and makes the code more readable. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-22cache: Add better cache loading sanity checksRichard Purdie
We've seen cache corruption where the pairs come out in a different order to the way we saved them for unknown reasons. Add better sanity checking to give a more user friendly error rather than a crash/traceback. Also allows the system to reparse and recover. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-22cache: Drop/simplify pointless type checkingRichard Purdie
Since we no longer have random data like version fields in these structures and we can assume any extra cache data subclasses our class, simplify the code. This is mostly reindenting after removal of the pointless type checks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-22cache: Improve versions fields handlingRichard Purdie
Firstly, don't store the versions fields in memory in the cache objects data store. This just complicates the code for no good reason. Secondly, write the version fields to all cache files, not just the core one. This makes everything consistent and easier. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-22cache: Correctly handle missing extra cachesRichard Purdie
If an "extras" cache file is corrupted, the system would not notice and later fail with errors about missing entries. Add a test for this which means we can fall back to re-parsing in those cases. [YOCTO #9902] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-22cache: Move the parsing message to a more logical placeRichard Purdie
Otherwise you can look at the log and wonder why parsing isn't happening when it really is due to other code paths. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21bitbake: implement idle timeout for xmlrpc serverEd Bartosh
Idle timeout can be specified either by -T/--idle-timeout option or by sessing BBTIMEOUT environment variable. Bitbake xmlrpc server will unload itself when timeout exprired, i.e. when server is idle for more than <idle timeout> seconds. [YOCTO #5534] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21bitbake: implement --foreground command line optionEd Bartosh
This option makes bitbake xmlrpc server to run in foreground. It should be useful for debugging purposes. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21bitbake-user-manual: Added new section on command-line executionScott Rifenbark
Needed a section on executing a list of task and recipe combinations. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21bitbake-user-manual: Applied review edits to "Recursive Dependencies"Scott Rifenbark
Added a minor tweak to the third paragraph to be more inclusive of the topic. Fixes [YOCTO #9970] Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Fixed typo in the BBDEBUG variable description.Scott Rifenbark
In the BBDEBUG variable description, the "-d" parameter needed to be "-D". Fixed it. Fixes [YOCTO #9950] Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Added review changes to BBCLASSEXTEND descriptionScott Rifenbark
Fixes [YOCTO #9909] Added some technical clarifications to the existing note to be clear about what is causing the limitations in this case. Applied some formatting fixed to for the use of the include word. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Added review changes to recursive section.Scott Rifenbark
Applied formatting to a "after" word that should be <filename>after</filename>. Also added a new clarifying paragraph in the "Recursive Dependencies" section. Fixes [YOCTO #9861] Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Addeds support for the Perforce FetcherAndrew Bradford
Added a new Perforce Fetcher section in the same spirit as the existing sections for other supported fetchers. Changes included the new section, removal of the bulleted item that mentioned this fetcher as an "additional" fetcher, and the creation of a new variable in the glossary named P4DIR. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Added clarifying note to BBCLASSEXTEND variable.Scott Rifenbark
Fixes [YOCTO #9909] Added a note that talks about how the BBCLASSEXTEND mechanism adds recipe variants. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Added a note providing examples of task dependenciesScott Rifenbark
Fixes [YOCTO #9861] In the "Dependencies internal to the .bb File" section, I placed a note providing more detail on how recipes are built regarding task dependency. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Updated dependency section for tasksScott Rifenbark
Fixes [YOCTO #9861] I updated the section on dependencies to give a couple of examples for task dependencies within a single recipe and dependencies for tasks between two individual recipes. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Updated the defaults for getVar and getVarFlagScott Rifenbark
There is no default. Fixes [YOCTO #9683] Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-21bitbake-user-manual: Added new section on command-line executionScott Rifenbark
Needed a section on executing a list of task and recipe combinations. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2016-07-20cooker: Fix incorrect dot file generationRichard Purdie
In the runqueue cleanup/conversion, "dep" was mistakenly used where "tid" should be leading to incorrect task-depends.dot files and causing general confusion. Fix this, its clearly incorrect looking at the code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20toaster-tests: add class SeleniumTestCaseBase for browser testsAníbal Limón
In order to reuse the Selenium helper outside a Django environment (for functional testing), add a new module containing the base class SeleniumTestCaseBase, which only inherits unittest.TestCase Add a class SeleniumTestCase with multiple inheritance of StaticLiveServerTestCase and SeleniumTestCaseBase to prevent existing tests from breaking. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20bitbake: main: implement server autostart featureEd Bartosh
If environment variable BBSERVER == 'autostart' bitbake will automatically load server if it's not running yet. If host and port are in bitbake.lock then bitbake tries to check if server is running and responses to commands and starts new server only if this check fails. [YOCTO #5534] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>