| Age | Commit message (Collapse) | Author |
|---|
|
Use the CustomImageRecipe generate_recipe_file_contents to generate the
recipe that we build from. Move creation of the dummy layer and recipe
object to the point of recipe creation as we need these objects before
the build time. Also update the methods to add and remove packages to
account for the CustomImageRecipe inheriting from Recipe.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Add function generate_recipe_file_contents to dump the custom image
recipe instance to a string for use either to push to the user as a
downloaded version of their custom image recipe or to use to generate
the recipe that we build.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
This adds the concept of CustomImagePackage this is similar to the way
layers and recipes work in that we have a set of data which is part of
the build history and a set of data which is part of the configuration
data that toaster uses to guide people in configuring their project. We
create a set of built_packages for every build but only create a package
for configuration purposes if we don't already have one, so that the
CustomImagePackage only ever contains a unique list of packages that are
available to be added and removed from a CustomImageRecipe.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
This table is used to track all the available packages in the current
toaster. Many of these packages belong to many CustomImageRecipes.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
This allows us to re-use the properties of a recipe for the custom image
recipes as well as re-using the existing templates and logic that deals
with recipe objects.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Add function that returns the Sum of the size of all the packages which depend on a package.
Access get_total_source_deps_size via a packages's dependency
manager.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Allow users of ToasterTable to manually trigger a refresh of the data.
This can be useful if an action has happened in-page and the data is now
invalid. Such as new data being added or removed from the model.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Move part of the functionality of the filter functions to the Table
widget. We don't need to implement it in each subclass.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Fall back to a 'n/a' string for the vcs reference, not all our source
has to be in a vcs and therefore it is legitimate for this to be none.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Update the reference to the base_recipe. It is now a Recipe object
rather than an intermediate AvailableRecipe object.
Therefore doesn't need an extra traverse down the object hierarchy.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
In order to use in other modules since is a common function
when needs to get proxies working.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The ID was omitted from the "edit columns" button on the
"projects", "all builds" and "project builds" tables when they
were converted to ToasterTable. This caused the QA tests to fail.
Reinstate the ID so the tests can identify the "edit columns" button
correctly.
[YOCTO #9051]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently any not catched exception in cooker causes bitbake
to hang because of not terminated children of CookerParser.
Long term solution would be to reimplement Cooker as a context
manager and terminate parser children in its __exit__ method.
Partial fix is to call CookerParser.shutdown in Cooker.shutdown in
hope that all Cooker exceptions are caught and shutdown method is
called.
[YOCTO #8900]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since people do copy and paste these things, clean up old syntax styles.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This shouldn't be in here, use a variable instead.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Anonymous functions are python functions, set the variable
flags as such so we can detect them and avoid expansion where
needed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently bitbake doesn't parse into data.expand() expressions,
relying on high level expansion of python code to handle this.
One of the tests does however test this works.
We don't really want to be doing string expansion on python code,
so specifically parse into expand() function calls so that when
the high level behaviour is tweaked, the self tests continue to
pass and that we do continue to handle expand() function calls as
best we can.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This sets the scene for removing the default False for expansion from
getVarFlag. This would later allow True to become the expand default.
On the most part this is an automatic translation with:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *`
There should be no functional change from this patch.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This fix a problem when checking out a commit that changes the submodules
previously checkout.
Example:
Recipe uses branch A and then it updates to use branch B, but branch B has
different submodules dependencies then what branch A previously had.
Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Clearing the umask when daemonizing is not the correct thing
to do, as it will create files writable by anyone by default.
For instance the pid file was being created with mode 777.
This could also potentially affect the sqlite database.
Better let the calling process decide on the umask.
[YOCTO #9036]
Signed-off-by: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current SIGTERM handler hungs the process instead of
making it exit. The problem seems to be that the handler thread
is not signaled to quit, so it stays there doing its work, as
it is not a daemon thread. Setting the quit variable fixes this.
While at it, to not use the SystemExit exception to terminate
upon SIGTERM but instead left the quit flag do its job. This way
the PID file is properly removed.
[YOCTO #9035]
Signed-off-by: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When daemonizing the PR service the -wal and -shm sqlite files were
being deleted, although they should never be. While the daemonized
process keeps the file descriptors open and thus a clean exit does
not loose any data, a power outage would loose all data in the WAL.
Removing these files also breaks sqlite collaboration between
processes and furthermore prevents taking proper backups without
stopping the PR service.
The reason this happens is that the DB connection is opened in
the initial process, before forking for daemonization. When the
DB connection is closed by the exiting parent processes it can
delete the -wal and -shm files if it considers itself to be the
last connection to the DB. This is easily fixed by opening the
DB connection after all forking.
[YOCTO #9034]
Signed-off-by: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I before E, except after C...
Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we're only going to parse one recipe, no point in starting
a large number of threads.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When running universe builds, we don't expect an error exit code for
provider warnings. Change the error messages to warnings in this case.
This deals with errors causing problems on our autobuilders amongst
other issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
findServerDetails function can be removed safely
from the source tree. Couldn't find any files
calling this function.
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The recent change to verify_checksum() to only show checksum warnings
if no checksums are supplied made it possible to simplify the logic a
bit more.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This means that when you view the process tree, the processes
have meaningful names, aiding debugging:
$ pstree -p 30021
bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593)
├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634)
└─{ProcessEQueue}(115591)
$ pstree -p 30021
bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335)
├─PRServ(117332)───{PRServ Handler}(117333)
├─Parser-1:2(117336)
└─{ProcessEQueue}(117331)
Applies to parse threads, PR Server, cooker, the workers and execution
threads, working within the 16 character limit as best we can.
Needed to tweak the bitbake-worker magic values to tell the
workers apart.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Being able to tell the bitbake processes apart is useful for debugging.
Add a helper function which allows this without making it a hard
dependency. Errors are ignored, this is just nice to have.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A newline is always appended to the function body when it's written
out, so strip any trailing newlines which may be there already.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix quoting of $BASH_COMMAND and avoid wrapping at 80 columns (the
script which follows is likely to contain some very long lines, so
line wrapping in bb_exit_handler() looks somewhat out of place).
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Update the help for BBMASK to reflect that it is now allowed to
contain multiple regular expressions. Also changed the examples for
BBMASK to be a bit more diverse.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Before, BBMASK was only permitted to contain one regular expression.
This made it hard to add to the BBMASK in multiple places as one was
supposed to separate the different regular expressions with a "|"
rather than with whitespace as is customary in BitBake variables.
Now one can specify any number of regular expressions in BBMASK. This
makes it possible to, e.g., mask out recipes in another layer from the
layer.conf file.
This also properly ignores any regular expressions that do not compile
(before an invalid regular expression would cause a ParseError in the
first bbappend file found stating that it was not a BitBake file...)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added more detail to the 'dir' flag to state the last
directory listed will be used as the work directory for the
task.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This directory no longer has setup.py as part of the directory.
I removed that line from the example in the
"Obtaining BitBake" section.
Fixes YOCTO #8623
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The core change here is to fall back to GET requests if HEAD is rejected in the
checkstatus() method, as you can't do a HEAD on Amazon S3 (used by Github
archives). This meant removing the monkey patch that the default method was GET
and adding a fixed redirect handler that doesn't reset to GET.
Also, change the way the opener is constructed from an if/elif cluster to a
conditionally constructed list.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add new runner options:
--run-all-tests: finds all tests, ignores config
--run-suite <suite> (from cfg)
Without arguments, run tests from current os section (config), e.g.:
1. ./run_toastertests
2. ./run_toastertests --run-all-tests
3. ./run_toastertests --run-suite darwin
Update toaster logging to meet QA CI requirements.
Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If one checksum is supplied to a SRC_URI, we really don't want to show
warnings about the other type which isn't present as one checksum
is really good enough for most cases.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster script run bitbake with --postread conf/toaster.conf.
It turned out that variables set this way don't influence the
build. Changing --postread to --read makes bitbake to read
variables earlier and should fix this issue.
[YOCTO #8781]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In certain circumstances it can be useful to get access to the world
targets list from a recipe in order to add dependencies on some or all
of the items in it. If a special function, 'calculate_extra_depends' is
defined in the recipe, and the recipe is to be built, then call it at
the right point before we calculate which tasks should be run. The
function can append items to the "deps" list in order to add
dependencies. This is not as tidy a solution as I would have liked, but
it does at least do the job.
As part of this change, the buildWorldTargets function was moved to
bb.providers to make it possible to call from taskdata.
Part of the implementation of [YOCTO #8600].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The rprovides maybe contain duplicated lines when parse again, we need
check it before add to cachedata.rproviders, similar to what we had done
to cachedata.providers.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is useful for newbie, for example:
$ bitbake rpm-build
ERROR: Nothing PROVIDES 'rpm-build'. Close matches:
pm-utils
rpm RPROVIDES rpm-build
[YOCTO #8881]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If variables are unset, the code simply doesn't expand them, there
aren't errors. If the code is a python expression, this can get a bit
messy, see the attached test case. The python expansion code sees the }
of the unexpanded value rather than the close of the python expression
and then raises a SyntaxError exception.
Ideally, we'd update the code to match pairs of brackets. I don't know
how to do that with the current regex and this is unfortunately a
performance sensitive piece of code. We also run the risk of breaking
existing code in OE-Core where there are "{" characters but not "}"
to close them (PKGE and PE).
Rather than raising the exception, matching the existing "just return
the expression" behaviour seems more consistent with the standard
variable behaviour.
This addresses an issue found in the recent image.bbclass code where
there are some variables we choose not to expand (TMPDIR/DATETIME).
This patch also adds a test case for this behaviour. It wouldn't preclude
improved bracket matching code in the future either.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A recent change in bitbake added filename/lineno information to the
parameters of bb.data.build_dependencies(). The codeparser tests
required a little adaption to the changes, adding the flags to the FOO
variable used in the tests.
The error seen when running the tests is a TypeError exception raised
in bb.codeparser:
TypeError: int() argument must be a string or a number, not 'NoneType'
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the resent change in rootfs creation code setVariable
rpc calls don't set variables for bitbake workers anymore.
Writen variables to toaster.conf should solve this issue.
[YOCTO #8910]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
which needs to run"
This reverts commit b22592af8145a8c7c4ada2fa7c1dee2e753eca46.
That commit isn't entirely clear about why this change is needed but
I do have a usecase where this breaks things. If for example you run
"bitbake X -c packagedata" and that packagedata is in sstate, you'd
expect this to work.
If sstate doesn't contain a do_populate_sysroot for a dependency, you
would still expect the command above to succeed and you would not
expect
it to rebuild that dependency. With the current code, this isn't what
happens. The code finds the sstate for do_populate_sysroot missing,
this makes the task "uncovered" and this in turn makes it unskippable.
The example I found with this was avahi-ui, where it would trigger
a build of libdaemon to obtain its populate_sysroot.
Since this behaviour seems completely incorrect, revert the older patch
and we'll address any issues that crop up as a result.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When an artifact download is requested, Toaster goes through a
convoluted series of conditions to decide which file to push
to the response. In the case of build artifact downloads for
command line builds, this caused an ugly exception, as command
line builds don't have a build request.
To simplify and catch more corner cases, remove the code which
fetches files via the build environment (we only support the local
build environment anyway). Then push all requests along a single
path, catching any missing file errors, missing object errors
or poorly-formed URLs in a single except clause which always returns
a valid response.
Also modify the text on the "unavailable artifact" page so it
says that the artifact doesn't exist, rather than it "no longer"
exists (exceptions may occur because an invalid artifact was
requested, rather than an artifact which was removed).
[YOCTO #7603]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Michael Wood
Aníbal Limón
Elliot Smith
Ed Bartosh
Richard Purdie
Felipe F. Tonello
Diego Santa Cruz
Phil Blundell
Sujith H
Peter Kjellerstedt
Andre McCurdy
Martin Samuelsson
Scott Rifenbark
Ross Burton
Daniel Istrate
Paul Eggleton
Robert Yang
Olof Johansson