| Age | Commit message (Collapse) | Author |
|---|
|
When "Preparing RunQueue" shows up you can expect to wait up to 30
seconds while it works - which is a bit long to leave the user waiting
without any kind of output. Since the work being carried out during this
time is divided into stages such that it's practical to determine
internally how it's progressing, replace the message with a progress
bar.
Actually what happens during this time is two major steps rather than
just one - the runqueue preparation itself, followed by the
initialisation prior to running setscene tasks. I elected to have the
progress bar cover both as one (there doesn't appear to be much point in
doing otherwise from a user perspective). I did however describe it as
"initialising tasks".
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Add the ability to enter a mode where only a specified whitelist of
tasks can be executed outright; everything else must be successfully
provided in the form of a setscene task (or covered by a setscene task).
Any setscene failure outside of the whitelist will cause the build to
fail immediately instead of running the real task, and any real tasks
that would execute outside of the whitelist cause an immediate build
failure when it comes to executing the runqueue as well.
The mode is enabled by setting BB_SETSCENE_ENFORCE="1", and the
whitelist is specified through BB_SETSCENE_ENFORCE_WHITELIST, consisting
of pn:taskname pairs. A single % character can be substituted for the pn
value to match any target explicitly specified on the bitbake command
line. Wildcards * and ? can also be used as per standard unix file name
matching for both pn and taskname.
Part of the implementation of [YOCTO #9367].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.
Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
In addition to the "currently running n tasks (x of y)" message, show a
progress bar for another view on how much of the build is left. We have
to take care to reset it when moving from the scenequeue to the
runqueue, and explicitly don't include an ETA since not all tasks take
equal time and thus it isn't possible to estimate the time remaining
with the information available.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Add support code on the BitBake side to allow sstate.bbclass in
OpenEmbedded to report progress when it is checking for availability of
artifacts from shared state mirrors.
Part of the implementation for [YOCTO #5853].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Implement progress reporting support specifically for the fetchers. For
fetch tasks we don't necessarily know which fetcher will be used (we
might initially be fetching a git:// URI, but if we instead download a
mirror tarball we may fetch that over http using wget). These programs
also have different abilities as far as reporting progress goes (e.g.
wget gives us percentage complete and rate, git gives this some of the
time depending on what stage it's at). Additionally we filter out the
progress output before it makes it to the logs, in order to prevent the
logs filling up with junk.
At the moment this is only implemented for the wget and git fetchers
since they are the most commonly used (and svn doesn't seem to support
any kind of progress output, at least not without doing a relatively
expensive remote file listing first).
Line changes such as the ones you get in git's output as it progresses
don't make it to the log files, you only get the final state of the line
so the logs aren't filled with progress information that's useless after
the fact.
Part of the implementation for [YOCTO #5383].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Add a class to help report progress in a task that consists of multiple
stages, some of which may have internal progress (do_rootfs within
OpenEmbedded is one example). Each stage is weighted to try to give
a reasonable representation of progress over time.
Part of the implementation for [YOCTO #5383].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
For long-running tasks where we have some output from the task that
gives us some idea of the progress of the task (such as a percentage
complete), provide the means to scrape the output for that progress
information and show it to the user in the default knotty terminal
output in the form of a progress bar. This is implemented using a new
TaskProgress event as well as some code we can insert to do output
scanning/filtering.
Any task can fire TaskProgress events; however, if you have a shell task
whose output you wish to scan for progress information, you just need to
set the "progress" varflag on the task. This can be set to:
* "percent" to just look for a number followed by a % sign
* "percent:<regex>" to specify your own regex matching a percentage
value (must have a single group which matches the percentage number)
* "outof:<regex>" to look for the specified regex matching x out of y
items completed (must have two groups - first group needs to be x,
second y).
We can potentially extend this in future but this should be a good
start.
Part of the implementation for [YOCTO #5383].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Since we're going to make some minor extensions to it, it makes sense to
bring in the latest version of python-progressbar. Its structure has
changed a little but the API hasn't; however we do need to ensure our
overridden _needs_update() function's signature in BBProgress() matches
properly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
If you're looking to find the latest console log repeatedly it can be a bit
tedious - let's just create a symlink just as we do with other logs to
make it easy to find.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Restructured EventWriter code to make it more readable:
- got rid of init_file method as it's called only once
- renamed exception variable e -> err
- renamed event variable e -> evt
- simplified main 'if' structure of send method
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
class EventLogWriteHandler is a simple wrapper class with only one
class member. Replacing it with namedtuple makes code less nested and more
readable.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is no need to remove output file as it gets rewritten by
open(self.eventfile, 'w') anyway.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
pickle converts python objects into the binary form that can't be
decoded to text and therefore can't be converted to JSON format.
Attempt to convert event objects raises this error:
TypeError:
b'\x80\x03cbb.runqueue\nrunQueueTaskSkipped\nq\x00)...
is not JSON serializable
Encoded pickled event objects to base64 to be able to convert data
structure to JSON.
[YOCTO #9803]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
EventLogWriteHandler object was created and used in
BBCooker.initConfigurationData.
This causes creation of multiple EventLogWriteHandler objects
and results in duplicated entries in the output event file
as BBCooker.initConfigurationData is called multiple times.
Added eventlogfile parameter to EventLogWriteHandler to avoid using
global variable DEFAULT_EVENTFILE.
Moved EventLogWriteHandler to the module level.
Created EventLogWriteHandler object in BBCooker.__init__ to ensure that only
one handler object is created.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The code is still a bit icky (and should be refactored to not use
Gdk.threads_enter/leave) but it should work about as reliably as
it did with Gtk+2.
Based on earlier patches by Maxin and Joshua.
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove unused css file that still had the bootstrap v2 license in it.
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In 2c88afb6 find_chains()'s taskid argument was renamed to tid but
taskid is still used as key to explored_deps dictionary. Use tid instead
of taskid.
Signed-off-by: George McCollister <george.mccollister@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In recipes which use the perforce fetcher, enable use of SRCREV to
specify any of: ${AUTOREV}, changelist number, p4date, or label. This
is more in-line with how the other fetchers work for source control
systems.
Allow p4 to use the P4CONFIG env variable to define the server URL,
username, and password if not provided in a recipe.
This does change existing perforce fetcher usage by recipes and will
likely need those recipes which use the perforce fetcher to be updated.
No recipes in oe-core use the perforce fetcher.
References [YOCTO #6303]
Signed-off-by: Andrew Bradford <andrew.bradford@kodakalaris.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Python 3 changed the return value of check_output to binary rather than
a string. This fix decodes the binary before calling splitlines, which
requires a string.
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The iterator used to create a search query refers to a
variable "x" which isn't set, causing an "'x' is not defined" error
and preventing table searches (on non-ToasterTables) from working.
Use the "field" variable instead, which contains the name of the
field to add to the query.
[YOCTO #9749]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
First parameter of traceback.format_exc is a 'limit' - a number
of stracktraces to format.
Passing exception object to format_exc is incorrect, but it works in
Python 2 as this code from traceback module works:
while tb is not None and (limit is None or n < limit):
Comparing integer counter n with the exception object in Python 2
always results in True. However, in Python 3 it throws exception:
TypeError: unorderable types: int() < <Exception type>()
As format_exc is used in except block of handling another
exception this can cause hard to find and debug bugs.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When parsing, we should reset the event handlers we registered when
done. If we don't do this, parse order may change the build, depending
on what the parse handlers do to the metadata.
This issue showed up as a basehash change:
ERROR: Bitbake's cached basehash does not match the one we just generated (
/media/build1/poky/meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb.do_unpack)!
This is due to the eventhandler in nativesdk.bbclass being run, despite
this .bb file not inheriting nativesdk.bbclass. The parse order was
different between the signature generation and the main multithreaded
parse.
Diffsigs showed:
bitbake-diffsigs 1.0-r2.do_unpack.sigbasedata.*
basehash changed from 887d1c25962156cae859c1542e69a8d7 to cb84fcfafe15fc92fb7ab8c6d97014ca
Variable PN value changed from 'nativesdk-buildtools-perl-dummy' to '${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}'
with PN being set by the event handler.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Another patch added migrations in the correct location (different date) so
these in the wrong directory can be removed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I'm not sure what possesed me when I wrote this code originally but its
indirection of everyting to use numeric IDs and position dependent lists
is horrific. Given the way python internals work, its completely and
utterly pointless from performance perspective. It also makes the code
hard to understand and debug since any numeric ID has to be translated
into something human readable.
The hard part is that the IDs are infectous and spread from taskdata
into runqueue and even partly into cooker for the dependency graph
processing. The only real way to deal with this is to convert everything
to use a more sane data structure.
This patch:
* Uses "<fn>:<taskname>" as the ID for tasks rather than a number
* Changes to dict() based structures rather than position dependent lists
* Drops the build name, runtime name and filename ID indexes
On the most part there shouldn't be user visible changes. Sadly we did
leak datastructures to the setscene verify function which has to be
rewritten. To handle this, the variable name used to specifiy the version
changes from BB_SETSCENE_VERIFY_FUNCTION to BB_SETSCENE_VERIFY_FUNCTION2
allowing multiple versions of bitbake to work with suitably written
metadata. Anyone with custom schedulers may also need to change them.
I believe the benefits in code readability and easier debugging far
outweigh those issues though. It also means we have a saner codebase
to add multiconfig support on top of.
During development, I did have some of the original code coexisting with
the new data stores to allow comparision of the data and check it was
working correcty, particuarly for taskdata. I have also compared
task-depends.dot files before and after the change. There should be no
functionality changes in this patch, its purely a data structure change
and that is visible in the patch.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Using positions in lists for flags is an odd choice and makes the code
hard to maintain. Maintaining a list is slow since list searches are
slow (watch bitbake -n slow massively with it) but we can use a set()
instead.
This patch uses python sets to maintain the lists of tasks in each state
and this prepares for changing the task IDs from being integers.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix a number of selectors which have changed after the port to
bootstrap3. Also fix the modal wait_until_visible and returning of the
text for the radio buttons in the modals for edit custom image and new
custom image on the build dashboard.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Use underscore instead of dash in the file name for the test settings.
Also fix the import of the settings module.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Fix the selectors after changes made for bootstrap3 and table links being
removed.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
- Update the string for recognising a duplicate image recipe
- Fix an incorrect relative import
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Get the ttTypeahead object on the input to see if it's been initialised
correctly.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
First parameter of traceback.print_exc and traceback.format_exc APIs is
a 'limit' - a number of stracktraces to print.
Passing exception object to print_exc or format_exc is incorrect, but
it works in Python 2 and causes printing only one line of traceback.
In Python 3 comparison of integer and exception object throws exception:
TypeError: unorderable types: int() < <Exception type>()
As these APIs are usually used in except block of handling another
exception this can cause hard to find and debug bugs.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Changed python to python3 in shebang to ensure that
manage.py is always run by python3.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Fix deprecation warning specify the parser used for creating the
BeautifulSoup object.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
The view code for downloading image files used the "r" flag
to read the file, then used the open file object to form the
HTTP response.
While this worked in Python 2, Python 3 appears to be more strict
about this sort of thing, and Django throws a UnicodeDecodeError
when a file opened this way is used in a response.
Open the file with the "b" flag (binary mode) so that Django can
correctly convert the binary file handle to an HTTP response.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Previously, we didn't specify a specific version of Selenium.
When upgrading to Python 3 and installing Selenium to work with it,
the JS unit test broke, as the report format produced by Selenium
had changed.
Modify the test so that it works with the latest Selenium report
format.
Add a note to the README that the given Selenium version should
be used to prevent unexpected test failures.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
The code previously imported urllib to make use of querystring
quoting, but was modified to support Python 3. During this
process, the reference to urllib was not fixed, which resulted
in table filters breaking.
Remove the reference to urllib (which is no longer imported)
and instead reference the imported unquote_plus() function.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Removed unneeded code as it causes the following error in Python 3:
TypeError: expected bytes, bytearray or buffer compatible object
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Python 3 complains about 'wa' mode this way:
ValueError: must have exactly one of create/read/write/append mode
Fixed by using 'a' mode.
[YOCTO #9584]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Decoded response content to str to pass it to json.load as it breaks
in Python 3 with this error:
TypeError: expected bytes, bytearray or buffer compatible object
[YOCTO #9584]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Python 3 doesn't have dict.viewkeys method, renaming
to keys().
[YOCTO #9584]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Explicitly use python3 so that the modules for python3 are checked.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
In the 'outcome' filter for tasks, remove the 'not applicable' option,
since it should not be exposed to users.
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
The explanatory message in the empty state of the packages built table
was missing some spaces.
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Make sure the current page is always highlighted in the left navigation.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
|
|
Port all the pages in the build analysis area to bootstrap version 3.
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Design change to have no links for non-named fields in BuiltPackagesTable.
BuiltRecipesTables and BuildTasksTable.
Additionally:
- Fix class and elements for Bootstrap3 in help text.
- Change title of InstalledPackages table to Packages Included.
- Change which columns are default shown/hidden columns.
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
'k' was replaced by 'key' at some point but not fixed in the
body of the loop. This caused a failure when the the query
was constructed for a filtered queryset, due to the variable
not being defined.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
In some pages of the UI, the UI tests failed because parts of
the UI which should be visible were being reported as not
visible by the Selenium PhantomJS driver.
On investigation, it turns out that PhantomJS uses a very narrow
default window. This meant that some parts of the UI were being
clipped and were thus not "visible" to the driver, causing test
failures (specifically, on the new custom image page).
Ensure that the window is maximized before running tests to
prevent this happening.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
|
Paul Eggleton
Ed Bartosh
Jussi Kukkonen
bavery
George McCollister
Andrew Bradford
Stephano Cetola
Elliot Smith
Ed Bartosh
Richard Purdie
Michael Wood
Belen Barros Pena