aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2016-05-20toasterui: add a new toasterui for experimenting withelliot/toaster/9475-db_locking_againElliot Smith
2016-05-20buildinfohelper: track usage of the db access APIElliot Smith
[YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: move all db access methods to ToasterApiClientElliot Smith
This will allow tracking of database accesses, so we can gather data about which models are created and how many of each. [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: use HTTP request to get model by primary keyElliot Smith
Modify buildinfohelper and the Toaster API client so that the BuildRequest is retrieved using an HTTP request. Note that Django serializers are used here as they do a better job of converting JSON into Django Model objects (which buildinfohelper currently requires). [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: use REST API to get default projectElliot Smith
Add requests module for making HTTP requests to Toaster's database API. Add an API client to fetch the default project over HTTP. Add a serializer for Project objects. [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: expose default project data via RESTElliot Smith
Put in the basic wiring for a REST framework which will provide the API for buildinfohelper to post build data to Toaster. [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: route other read/write operations through ORMWrapperElliot Smith
Move other operations which trigger SQL queries to be run against the database so that they reside in ORMWrapper. This should mean that buildinfohelper no longer does any read/write on the db. [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: move filter and bulk_create to ORMWrapperElliot Smith
Instead of calling filter() and bulk_create() directly on Django models, route these calls through ORMWrapper. This is to support moving database access to HTTP calls later. [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: funnel create operations through ORMWrapperElliot Smith
Rather than call create() on model managers, route all create operations through a method on ORMWrapper. This lays foundations for moving model creation to HTTP requests. [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: move saves to a methodElliot Smith
Rather than saving directly on model instances, route all calls to save() through a method on ORMWrapper. This lays more foundations for converting these calls to HTTP requests later. [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19buildinfohelper: funnel read operations through methodsElliot Smith
Instead of calling read methods directly on model classes, route them through methods on ORMWrapper. This lays foundations for converting them into HTTP requests later. Also remove our own memory-based caching and use Django instead (as Django caches queries anyway). [YOCTO #9475] Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: tests builds Add SSTATE_MISS as a valid condition for tc=833Michael Wood
Task.SSTATE_NA and Task.SSTATE_MISS are both valid conditions for the condition that a Task.OUTCOME_COVERED and Task.OUTCOME_PREBUILT. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: tests build Add a test for a build of core-image-minimalMichael Wood
This is a port of the oe self test to the django test framework from oe-core meta/lib/oeqa/selftest/_toaster.py Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: tests Add a BuildTest helper classMichael Wood
Add a helper class for running build tests. Subclass this and call the build method to get setup for running tests on the resulting data from a build. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: Add a specific test settings fileMichael Wood
When running certain tests we want a particular database specified. When bitbake toaster ui is being tested pass it these test settings so that it uses the same database as the unit tests running. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: runbuilds move the execution sequence out of the poll loopMichael Wood
Move the execution sequence for a build out of the polling loop and into it's own fuction. This means that we can call the function on it's own if we just want to trigger one build rather than infinite polling. This is something needed for the build tests. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: Remove DATABASE_URL being passed around as an environment varMichael Wood
We don't need to pass the DATABASE_URL around and read it back if we setup the django framework in the correct way. We make the default sqlite database path a full path so that the database isn't being assumed to be in CWD. Also add some more useful comments on the database settings. This is preparation work to migrate the build tests and be able to trigger builds on differently configured databases. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19bitbake: toaster-tests: tests for project configSujith H
Add basic tests to validate the value user types in the text box for IMAGEFS_TYPES. Added a test case to show the checkbox get automatically selected when user types value available in the check list. Added a test case to verify if the check box is enabled then the text box should also get updated accordingly. [YOCTO #7828] Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: projectconf Small tweaks to IMAGE_FSTYPES formBelen Barros Pena
Add a label to clarify the purpose of the first input field, replace <p> with <label> for the second set of controls, and give some top margin to the validation message. Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-19toaster: handle multiple imagefs typesSujith H
This functionality helps users to add custom image fs types available other than the checkboxes in the UI. User can add imagefs types in the text box and use them in the build. [YOCTO #7828] Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-16toaster: toastergui tests Fix toastertable testsMichael Wood
After clean ups remove api assumptions. Our table data often contains html snippets to display certain things such as dependency pop overs or simply links to other parts of the UI. Take these into account when testing the values of the table data. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-16toaster: toastertables: Clean up and fix up a number of itemsMichael Wood
- Remove the unused 'computation' field - Remove the code to try to make the tables behave like an api - Remove custom JSON encoder in favour of DjangoJSONEncoder - Simplify get_data and add comments - Add exception type instead of using generic Exception - Clean up python style warnings Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-13toaster: fix progress bar in MySQL environmentElliot Smith
When using MySQL, the project builds info delivered by MySQL differs from that delivered by SQLite: the former returns text values from the enumeration for Build outcomes, while the latter returns the integer value. This causes the progress bar JS to break, as it is expecting outcome strings. Modify the recent_build() method to include an outcomeText property for each Build object, then use this in the conditionals in the progress bar JS. [YOCTO #9498] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
2016-05-12toaster: don't show "Rebuild" button for cancelled cli buildsElliot Smith
[YOCTO #8515] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
2016-05-12toasterui: capture keyboard interrupts the same way as knottyElliot Smith
knotty captures two levels of keyboard interrupt: a single interrupt or two interrupts in a row. These then trigger stateShutdown and stateForceShutdown respectively. toasterui doesn't have an equivalent way of capturing interrupts and using them to shut down bitbake. Now that we are no longer using knotty + XMLRPCServer for our command line builds (since switching to per-project build directories), we see some odd side effects of this, such as builds continuing after they have been interrupted on the command line. Bring toasterui in line with knotty (copy-paste most of the code in knotty.py which deals with interrupts) so that a keyboard interrupt actually shuts down the bitbake server (if not in observe only mode). Additionally use the cancel_cli_build() method to set the Build status to CANCELLED in Toaster's db when we get keyboard interrupts. This means that builds interrupted on the command line show as cancelled (same as if they'd been cancelled from the Toaster UI), as specified in the UI designs. [YOCTO #8515] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
2016-05-12buildinfohelper: add method to set current build as CANCELLEDElliot Smith
This will be used from toasterui to cancel the current command-line build when a keyboard interrupt is captured. [YOCTO #8515] Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
2016-05-12toaster: tweaks to recipe file downloadsBelen Barros Pena
* In the custom images table, show the recipe file and download icon only when the recipe file exists and can be downloaded. * Also in the custom images table, make sure the download icon tooltip shows on hover. * In the custom image details page, show the correct icon (download) next to the recipe file in the right hand column. * In the custom image details page, show the recipe file and download icon in the right hand column only when the recipe file exists and can be downloaded * Also in the custom image details page, simplify the help text we show when the 'download' button is disabled: it gave so much information about what's actually happening under the hood that it was a bit hard to follow. Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-12toaster: add DL_DIR and SSTATE_DIR to oe toasterconfEd Bartosh
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com>
2016-05-12toaster: testsjs Add call back to append elements on completion of testsMichael Wood
When the tests are complete add some marker elements to the test page DOM which can then looked at for a selenium browser test. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
2016-05-11bitbake: toaster: tests browser Add test for creating a projectMichael Wood
Add browser tests for creating a project and test validation of duplicate project names. (Bitbake rev: 8055fbe840db426d6859ee2248f86abd44244b30) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: toaster: models Add missing import sysMichael Wood
(Bitbake rev: b2fba7a0820d77e1076c05b8ffcd960d4fc883a5) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: toaster: ui handles duplicate project name in project pageSujith H
When already existing project name is typed by user, the ui pops up message regarding the existance of the project name. When an existing project is typed the save button will be disabled. Else user can proceed ahead by modifying the project name. [YOCTO #7005] (Bitbake rev: 05ddf48cda6690adab4c097b16387578523e751b) Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: toaster: ui handles duplicate project name in new project pageSujith H
When already existing project name is typed by user, the ui pops up message regarding the existance of the project name. [YOCTO #7005] (Bitbake rev: 83e5be7e74850f1bb019668de07f3f745063fe38) Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: toaster: projectNameValidation API addedSujith H
The projectNameValidation API would help users to validate if a project name exists or not. This API is added to libtoaster. [YOCTO #7005] (Bitbake rev: 3b1843553f23d78f1ddfec9f7865895ee42356a3) Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: runqueue: Fix missing fakeworker under dry runRichard Purdie
We shouldn't try and use fakeworker when performing a dry_run. This makes the core match the other fakeworker execution points. (Bitbake rev: 49bea821a2edad5e19c3a566d1a80c23718dede9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: lib/bb/utils.py: Fix explode_dep_versions2() determinism issueRichard Purdie
When we pass data into explode_dep_versions2(), we need to result to be able to be processed in a deterministic way so that we end up with consistent hash values. This means we need an ordered structure rather than an unordered one. To do this, return an OrderedDict() rather than a dict(). (Bitbake rev: 0737e003ca549d08a7dfe13452ae982f2e11fecd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: bin, toaster: Fix print and exception syntaxRichard Purdie
This updates the print "" syntax to print() and fixes some exception handling syntax such that its compatible with python v2 and v3. (Bitbake rev: 58304fcce9727fd89564436771356c033ecd22a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: tests: assertEquals -> assertEqualRichard Purdie
The preferred form is assertEqual, assertEquals is deprecated and not present in python v3. This is v2.7 safe. (Bitbake rev: b60261bf8ade14aca31238b50c243c01adcabc59) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: bitbake: Update logger.warn() -> logger.warning()Richard Purdie
python deprecated logger.warn() in favour of logger.warning(). This is only used in bitbake code so we may as well just translate everything to avoid warnings under python 3. Its safe for python 2.7. (Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11oeqa/lic-checksum: Update after recent LIC_FILES_CHKSUM changesRichard Purdie
The check now runs at populate_lic time so change this from configure. We also need to set a SRC_URI so that the check triggers. (From OE-Core rev: a5970809a2f01dbd152684266e2a3d946d896620) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11devtool: Fix build-sdk when pn doesn't match filenameRandy Witt
If an image with the filename foo.bb could be built using the name "bar" instead, then build-sdk would fail to create the derivative sdk. This was because the code assumed that the file name matched the target, which is not necessarily the case. (From OE-Core rev: d58a326b6960be14b8a049253559aec9582b7d0d) Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11Drop unneeded LIC_FILES_CHKSUM valuesPaul Eggleton
With the changes to insane.bbclass we don't need LIC_FILES_CHKSUM to be set for recipes that don't actually pull in any source. (From OE-Core rev: fc14e794c53f94158a5e6d6a8644656875639d0c) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11classes/insane: do not force LIC_FILES_CHKSUM unless SRC_URI is setPaul Eggleton
Recipes such as images and packagegroups don't actually fetch or build any source, so there's really no point having LIC_FILES_CHKSUM set. Forcing users to set it (as we have done for images inheriting image.bbclass) just makes things more difficult for the user for no discernable benefit. The easiest way to adjust this check is simply to skip it if SRC_URI is not set since this is a pretty good indicator that no source is being pulled in. (From OE-Core rev: c269547ae8e90a78349f6003385137e4145e145f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11lib/classextend: Fix determinism issueRichard Purdie
The ordering of dependency variables needs to be deterministic to avoid task checksums changing. Use an OrderedDict to achieve this. (From OE-Core rev: 855a2d21503856af392ab2d54ccfa270505ba142) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11update-alternatives: Fix determinism issueRichard Purdie
getVarFlags returns a dict and there is therefore no sort order. This means the order of the X_VARDEPS_X variables can change and hence the task checksums can change. This can lead to rebuilds of any parts of the system using update-alternatives and their dependees. This is a particular issue under python v3. Add in a sort to make the order of the variables deterministic. (From OE-Core rev: ecd1bfed5534f83b775a6c79092c04bd13c3af0a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11image: Fix IMAGE_FEATURES determinism issueRichard Purdie
remain_features uses a dict which means the order is not deterministic. This can lead to the task hash changing depending on the state of the memory at parse time. This is particularly noticeable under python v3. Since the dict is helpful in constructing the data, pass the data through sort() so the order is always deterministic. (From OE-Core rev: b08344e28dd33e3af5596007b11185d04fce255e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake.conf: add default for IMAGE_FSTYPES_DEBUGFSRoss Burton
If debug filesystem generation is enabled but this isn't assigned then the generation code throws exceptions. (From OE-Core rev: 0a1b02fab0e2604cd55ea6f45d764a864599213a) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11metadata_scm.bbclass: Do not assume ${COREBASE} is a Git repoPeter Kjellerstedt
The functions base_detect_revision() and base_detect_branch() try to extract SCM meta information from the path returned by base_get_scmbasepath(), which currently returns ${COREBASE}. However, making the assumption that ${COREBASE} contains SCM meta information can be false. It is true for Poky, but not necessarily other environments. A better option is to look for the SCM meta information based on the meta layer. Since this works as expected for Git but not SVN, the call to base_get_metadata_svn_revision() from base_detect_revision() was also removed. This is not expected to affect anyone (partly based on the comment in base_get_metadata_svn_revision()). (From OE-Core rev: 53fd0a4a37023642a770a9fbf3cd5511d3c82af7) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11lttng-tools: filter random filename of ptest outputDengke Du
When run the ptest of lttng-tools, it produced many random filename when the tests passed, the output confused QA analysis, so we need to filter the ptest output if tests passed and add up the passed and failed tests. NOTE:The tests invoked the run.sh twice, so it output like this: ... FAIL:... unit_tests statistics total pass: 133 tests passed! total fail: 5 tests failed! ... FAIL:... fast_regression statistics total pass: 1904 tests passed! total fail: 202 tests failed! (From OE-Core rev: 29a8c45be2862be02afe2ebbc5c026a42f351990) Signed-off-by: Dengke Du <dengke.du@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11feature-arm-neon.inc: restore vfpv3-d16 supportAndré Draszik
Commit 6661718 (feature-arm-{neon,vfp}.inc: refactor and fix issues) effectively changed the gcc -mfpu= option from -mfpu=vfpv3-d16 to -mfpu=vfpv3d16, which gcc doesn't understand. Restore the original value. After doing that, we also need to adjust ARMPKGSFX_FPU which should contain the same value without dash '-' as it is used that way throughout. (From OE-Core rev: 972b4fc459258572eeaad8af91e48ee9f0acade7) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>