| Age | Commit message (Collapse) | Author |
|---|
|
Fixed migrations by running manage.py makemigrations --merge
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
- Organise the imports into logical groups
- Fix 80 col wrapping
- Remove catch all exceptions
- Log to the toaster log
- Use QuerySet functions such as .first() and Q()
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Add handlers to make sure we remove the BuildEnvironment LOCK when we
have cancelled a build.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Don't count CANCELLED builds when returning the number of builds.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Add the cancel build button to the mrb section template and add the event
handlers to cancelABuild.
Also clean up the calls to startABuild to use the updated libtoaster methods
and to make the code consistent with it's cancelABuild counterpart.
Co-Author: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Exclude cancelled builds from showing in the builds table
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
|
|
When a build is cancelled the build (action) is complete if it has been
caused the request being cancelled then update the build outcome
accordingly.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
To accurately reflect the state of a build request we also need a
cancelling state. This is set when we've started a build and then for
whatever reason cancel it, cancelling is not instantaneous so we have
this state to indicate that a cancel is in progress.
Also add a state transition guard. As the state of a BuildRequest can
currently be modified by three processes; Toastergui,
Runbuilds/bldcontrol and the buildinofhelper we cannot say for sure
which process will be running at the time of cancellation so in order to
avoid one of these processes making an incorrect transition only allow
transitions of state to increase.
e.g. CREATED -> QUEUED -> INPROGRESS
And to ignore such requested changes such as
INPROGRESS -> CREATED
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
A new state CANCELLED is introduced to, distinguish
the state of build.
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
|
|
Remove getBBController function from BuildEnvironmentController.
The constructor of BitbakeController is updated appropriately so that
call can be made to connect to running server.
The call to startBBServer is removed from bbcontroller and handledin
localhostbecontroller.
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
|
|
Update the implementation of startABuild and cancelAbuild to reflect
changes to the backend api. We now have a dedicated endpoint to make
calls into so add this url to libtoaster.ctx and allow passing null in
as a url value to indicate that we want to use the current project
Also:
- Fix some documentation comments
- Add the convenience of passing in an array of targets to startABuild
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Update the implementation of the backend api for cancelling builds with
the new cancelling BuildRequest state and cancelled Build state.
Also added some docstring about general usage.
Co-Author: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Move the backend xhr implementation of the build request changes
into it's own file and out of the ToasterTable definition.
It used to live in the views.py but in a hope of starting to collate logical
groups of views move this to a new file called api.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Add forceShutDown function to BitbakeController class. This function
provides a mechanism to cancel the build from toaster. An API which
can be used safely to cancel build and hence shutdown running bitbake
server.
[YOCTO #6787]
Signed-off-by: Sujith H <sujith.h@gmail.com>
|
|
Bitbake server is used only during the build. There is no need
in keeping server running after the build.
Running bitbake -m in the subshell after the build should stop
the server.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Added 'nowait' parameter to _shellcmd method to support
running chain of commands in a subshell.
This is going to be used to stop bitbake server after
the build.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Reimplemented triggerBuild method to support one build directory
per project:
- start bitbake server from the cloned repository
- don't run observer
- run bitbake build directly instead of triggering it
through xmlrpc
[YOCTO #7880]
[YOCTO #9058]
[YOCTO #8958]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Removed updating of bblayers.conf. It will be done in runBuild method.
Changed return value: return list of layers.
Removed _updateBBLayers method.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Called triggerBuild with brbe parameter instead of adding TOASTER_BRBE
variable to the database and fetching it in triggerBuild.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
This API is quite dangerous as it removes build directory.
It's not used anywhere and most probably will not be used in future
as toaster is going to have one build directory per project.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
We still will have to run bitbake server, but it will be done
different way and the code will be in triggerBuild function.
Removed startBBServer API from BuildEnvironmentController and
LocalhostBEController classes.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
buildinfohelper.brbe is lost when buildinfohelper is closed.
This causes incorrect report of brbe when build is done.
Saved brbe attribute before closing buildinfohelper and used
it to report correct brbe.
Got rid of useless and confusing 'ToasterUI build done 1'
log message.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
DepTreeGenerated event doesn't contain 'providermap' data in jethro.
Modified buildinfohelper to handle events without this data. This
should make it possible to handle jethro events coming from jethro
bitbake server by the latest buildinfohelper.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Return value of self.BBServer.registerEventHandler differs between
jethro and master. To be able to build jethro toaster should be
able to communicate with jethro bitbake server i.e. it must work
with both old and new registerEventHandler call.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
In current toaster code BRBE(build request:build environment) value
is passed from toaster to buildinfohelper through the 'SetBRBE' event.
Passing it through environment variable is easier as it doesn't
involve rpc communication between toaster and bitbake server.
It also eliminates the need in running bitbake observer process.
Added parameter 'brbe' to BuildInfoHelper.__init__
Used environment variable TOASTER_BRBE to set brbe for
buildinfohelper object.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Set BITBAKE_UI variable to 'toastergui' for command line builds
to use toasterui as a default ui module for bitbake.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
noui command line option doesn't makes sense anymore as toaster doesn't
run bitbake. It should be safe to to remove it.
The purpose of this option was to skip running bitbake observer process.
This was never used before as it's not possible to run toaster build
without running observer.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
From now on toaster script will not run bitbake server.
It will be started by runbuilds and stopped after the build.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
We were previously ignoring pkgs that came in from a setscene like from
an sstate mirror). With this patch we can use pkgs than come from sstate
for image customisation as well.
Also remove unused variable.
[YOCTO #9137]
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
If a build fails because of a bitbake error occurring before the
BuildStarted event fires, we do not generate a Build object
for command-line builds. This means that failed command-line builds
don't appear in Toaster at all.
To resolve, split build creation into two steps:
1. Just before buildTargets() is invoked on the XMLRPC server: create
the base Build object. Note that as soon as a Toaster-triggered
build starts, targets are added to it; but this event is the earliest
point when task and targets are available for command-line builds.
(This requires a new TargetsAcquired event to be fired by the XMLRPC
server when the buildTargets() command is called.)
2. BuildStarted event: add any layer information to either type of build
(command-line or Toaster-triggered).
Note that the build_name property cannot be set until BuildStarted,
as it is not available until then, which could cause problems
for creating Build objects earlier; however, this property is
redundant, as it's never used anywhere in Toaster, so it has been
removed (along with any functions which refer to it).
[YOCTO #8440]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
It looks like the master branch will no longer build with the Fido
release, so remove all references to fido and make sure that the file
sets up local, master and jethro releases.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
|
|
It looks like the master branch will no longer build with the Fido
release, so remove all references to fido and make sure that the file
sets up local, master and jethro releases.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
|
|
Instead of trying one time with a timeout of 20 seconds try 4 times with
a timeout of 5 seconds, to account for a slow server start.
(Bitbake rev: 4a7fe63126dd8177baa5ad21e59e0bebeea8c596)
Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When returning the compatible layers make sure that we are only
listing: All the layers which are for this release && configuration
layers (i.e. aren't part of the build history) and which aren't an
imported layer OR are this project's imported layer(s).
[YOCTO #8944]
(Bitbake rev: de8baedaccb451c12bc3f642449db3f64aed6bf7)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:
* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.
There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.
[YOCTO #8842]
(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When user enters a non-ascii character in the project
name of toaster, the build doesn't get triggered.
Use force_bytes to fix this.
Also deal with non-ascii project names when logging the
build request in runbuilds.
[YOCTO #9071]
(Bitbake rev: b6141c4d170885d3bdf63074afcb1e41fde0a8f0)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For some reason, the values for SRC_URI[md5sum] and SRC_URI[sha256sum]
were not being expanded. That lead to the following code not working
as expected:
SRC_URI = "http://.../${PN}-${PV}.tar.gz"
MD5SUM = "123abc..."
SHA256SUM = "abcd1234..."
SRC_URI[md5sum] = "${MD5SUM}"
SRC_URI[sha256sum] = "${SHA256SUM}"
(Bitbake rev: ba011470df0ea8bd89f01c0b02ec4b3969e60ce7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
XMLRPCServer.single_use attribute was always set to False.
This caused xmlrpc server to keep running after build is done as
BitBakeServerCommands.removeClient only shuts down server if its
single_use attribute is set to True.
(Bitbake rev: 0a60b0928a0a746a60d2c2f294ff1903963c7086)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Without this you get a rather odd traceback instead of the proper
exception message.
(Bitbake rev: 2fe1826d3077eeda6cde433d3a1e6620f74e08dd)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The output of "npm view dependencies" isn't entirely JSON if there are
multiple results, but the code here was just discarding the output if
the entire thing didn't parse as JSON. Split the output into lines and
iterate over it, parsing JSON fragments as we find them; this way we end
up with the last package's dependencies since it'll be last in the
output.
Digging further, it seems that the dependencies field reported by "npm
view" also includes optional dependencies. That wouldn't be a problem
except some of these optional dependencies may be OS-specific; for
example the "chokidar" module has "fsevents" in its optional
dependencies, but fsevents only works on MacOS X (and is only needed
there). If we erroneously pull in fsevents, not only is it unnecessary
but it causes "npm shrinkwrap" to throw a tantrum. In the absence of a
better approach, look at the os field and discard the module (along with
any of its dependencies) if it isn't for Linux.
As part of this, we can reduce the calls to npm view to one per package
since we get the entire json output rather than querying twice for two
separate fields. Overall the time taken has probably increased since we
are being more thorough about dependencies, but it's not quite as bad as
it could have been.
(Bitbake rev: 436d67fe7af89ecfbd11749a6ae1bc20e81f2cc8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"2 || 3" is a valid version specification for a dependency in an npm
package.json file, but of course that looks like something else when
sent to a shell. Quote the version value to avoid this.
(Bitbake rev: bea0246831a46d943d2e27d6b38f6e498bd3413c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Commit 8b81bb56c69aabdea984352f8e267a9783c0bdbc was accidentally merged.
The DL_DIR piece was simply incorrect and should be removed.
The patch commit message should have mentioned that the changes were
to update populate_sdk_ext after the changes to uninative now the
download is placed into a specific directory in DL_DIR. We also
need to specify the uninative tarball checksum.
(From OE-Core rev: be177739b46d3296c0c0c01310b499ffc6782d43)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"npm shrinkwrap" creates a file that ensures that the exact same
versions get fetched the next time the recipe is built. lockdown is
similar but also includes sha1sums of the modules thus validating they
haven't changed between builds. These ensure that the build is
reproducible.
Fixes [YOCTO #9225].
(From OE-Core rev: 277377f13b2b771915eb853e336ca24b84523ed1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.
devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.
(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the user specifies an npm:// URL then the fetcher needs npm to be
available to run, so check if it's available early rather than failing
later.
(From OE-Core rev: a08d12ad867c292f7474731a0fe5e51e712446d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.
Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.
(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Matching license texts directly to md5sums only goes so far. Some
licenses make the copyright statement an intrinsic part of the license
statement (e.g. MIT) which of course varies between projects. Also,
people often seem to take standard license texts such as GPLv2 and
reformat them cosmetically - re-wrapping lines at a different width or
changing quoting styles are seemingly popular examples. In order to
match license files to their actual licenses more effectively, "crunch"
out these elements before comparing to an md5sum. (The existing plain
md5sum matching has been left in since it's a shortcut, and our list of
crunched md5sums isn't a complete replacement for it.)
As always, this code isn't providing any guarantees (legal or otherwise)
that it will always get the license correct - as indicated by the
accompanying comments the LICENSE values it writes out to the recipe are
indicative and you should verify them yourself by looking at the
documentation supplied from upstream for the software being built if you
have any concerns.
(From OE-Core rev: 553bb4ea5d51be5179e7d8c019740cf61ece76ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For example, this picks up a file named MIT-LICENSE.txt.
(From OE-Core rev: 103b4d26b340cbdf70bf43906e293f3497671fdc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make the following improvements to mapping items specified in
AC_CHECK_PROG, AC_PATH_PROG and AX_WITH_PROG to recipes/classes:
* Produce a map of native recipe -> binary for all binaries currently in
STAGING_BINDIR_NATIVE and use this when mapping items
* Add some more entries to the class map
* Ignore autotools binaries since they are covered by the inherit of
autotools
* Ignore coreutils-native since that would almost always be a bogus
dependency
(From OE-Core rev: 5614c5ae6a004d4367eccc34dd3cc7ee61fb7e57)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow for whitespace in appropriate places, and ensure we match all
whitespace chars not just the space character.
(This fixes extracting dependencies from tmux's configure.ac, for
example.)
(From OE-Core rev: 63524ac8093b734aa4f29f4ea47bcc036f748314)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Ed Bartosh
Michael Wood
Sujith H
brian avery
Elliot Smith
Belen Barros Pena
Lucas Dutra Nunes
Peter Kjellerstedt
Paul Eggleton
Richard Purdie