| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit e7b32fcd4df4801ebf152de2bb3d5cd46d39989c.
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(From meta-yocto rev: 829e48e3cd3c3dd90855750ef469cb65f93f0d20)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(From OE-Core rev: 2186992e29a5e4625723c3dab88e69159f365f3b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We sometimes see exceptions from code seeing the hashserv DB files
being removed at directory cleanup time. Add a check to ensure the
hashserv has written the data base journal (and hence likely exited)
before cleaning up.
This will hopefully avoid errors like:
Traceback (most recent call last):
File "[...]/meta/lib/oeqa/sdk/buildtools-cases/build.py", line 30, in test_libc
delay = delay - 1
File "/usr/lib/python3.6/tempfile.py", line 948, in __exit__
self.cleanup()
File "/usr/lib/python3.6/tempfile.py", line 952, in cleanup
_rmtree(self.name)
File "/usr/lib/python3.6/shutil.py", line 486, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/lib/python3.6/shutil.py", line 424, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/lib/python3.6/shutil.py", line 444, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/lib/python3.6/shutil.py", line 442, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'hashserv.db-wal'
(From OE-Core rev: 4a9f77a4e09720f9c7bb97ac4480059caac32f8c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move the "exit with a fatal error" code into a common function
and also improve the confusing variable to something at least self
contained in the same code segment.
This will hopefully allow the insane code return value passing to be
cleaned up too.
(From OE-Core rev: 3889e8ab4bf47ad02e688523916a1b4d7730d378)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes. Add import statements and convert callers
to use oe.qa. prefix.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793
[RP: Drop imports, extend to other class files]
(From OE-Core rev: 06515cee8d0794e0469f331dcd06ac678c7e1c79)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
individual recipes, the distro or other configuration to determine
whether a missing licence should be treated as a warning (as it is now)
or as an error.
oe.qa.handle_error isn't immediately fatal, so track the overall sanity
state and call bb.fatal if required at the end to ensure that the task
really fails. If only bb.error is used then do_populate_lic isn't re-run
on subsequent builds which could lead to the error being missed.
It seems odd for the license- error classes to be listed in
insane.bbclass but implemented in license.bbclass. All recommendations
for somewhere else to put them gratefully received.
(From OE-Core rev: 28e8cd4f11d673ad42ac3d057e2a018b1bf9ea8d)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In 2017 we added changes to pass the BUILD_CFLAGS into the kernel
via BUILD_CC. This isn't really correct and the upstream kernel now has
places to pass build cflags, ldflags and more. Update our kernel
make flags to correctly use the kernel's variables. This addresses
concerns raised by kernel developers.
If this breaks some usecase please report it so we can work out how
to fix it properly.
(From OE-Core rev: 994dde2857f49ee5d90e3faefa9a2e4d9a24e9b3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We want things to be reproduicble and the variable doesn't really change
much any more. Drop the remaining uses and make those code paths always
active.
(From OE-Core rev: 107a37e5389d8ab4c3761aaf1bcff55c2cde1d44)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Change the "binary reproducibility" configuration within the kernel to
work off a separate variable, defaulting to reproducible builds.
This allows kernel developers wanting timestamps in their images to enable
it easily and clearly without changing the rest of the reproduciblity
code which they likely don't need to change anyway.
(From OE-Core rev: 13912995369f8355562d8eeeaa479d57d5979a85)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reproducibility is here to stay and needs to be part of our default workflow.
Move the remaining code to base.bbclass so it is always a first class citizen
and it is clear people need to be mindful of it.
(From OE-Core rev: 0c0d632e61957638516e0cbd98d075870ef99f80)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The reproducibility code was originally developed as separate
standalone class but development is no longer experimental and
the code is widely tested and used by default for poky. Reproducible
builds are the direction we need to take as a project.
Transition the core variable definitions to bitbake.conf as part of
a move to make these part of the default workflow.
This also helps reduce test matrix complexity as there is now one
code path.
(From OE-Core rev: e87d01a63abbf7cd96a477c9bae6d68e1fd6cec5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To try and avoid parse/memory overhead of functions within bitbake,
move the bulk of the reproducibility functions to the function library.
(From OE-Core rev: 6e129a26aa299729553c7de48602a13daa380b66)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We now pass compiler options that mean the compiler will adjust these values to
SOURCE_DATE_EPOCH, the compiler warnings are now unneeded.
(From OE-Core rev: c4932ca89cec32495d9affb6061bd4e822ac0c63)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This gives more meaningful errors.
(From OE-Core rev: 8bce75fb25b4e50b1c4c5680acccffb82bac0089)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if you have something like:
SRC_URI = "file://foobar;subdir=${S}"
and a file like:
foobar/1/somefile
and then move it to:
foobar/2/somefile
the task checksums don't reflect/notice this. The file-checksum fields
encode two pieces of data, the file path and whether or not the file
exists. Changing the code which uses these fields is problematic.
We can however add a "/./" path element which means "include the bit
after the marker in the checksum" which the path walking code can use
to mark which bits of the path are visible to the fetcher.
I'm not convinced this is great design but it does appear to work.
(Bitbake rev: 13f1126626300de88376c6a83fb92bf3e4140dd6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We occasionally see bitbake-worker failing and from the logs, an unpickle error
occurs. Add more debug so we can further debug this next time it fails.
[YOCTO #14595]
(Bitbake rev: b9b6951ea082ab030c90f751577c7fefe5858766)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We've seen races where the socket may be gone but the server is still writing
out it's database. Handle that case too to avoid cleanup tracebacks.
[YOCTO #14440]
(Bitbake rev: b70b602edb634f99a24b2cd39dca45eba2c3079c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These patches are already available in 3.0
(From OE-Core rev: 063d085534b7b3659c5721228bb58f4e8115b5ee)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(From OE-Core rev: f5a90df08631117cd66bec68cb1851db96822bb8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow a user to validate sstate objects against a list of keys, instead
of just any known key in the user's keychain.
(From OE-Core rev: 52ba0c5e6e2e3d5d01dc3f01404f0ab1bb29b3b5)
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Ensure the jinja2 module is available during bringup.
(From OE-Core rev: c46a6ec91bd40a458cb0ef5ec84bc0cc274d9cef)
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch switches the compressor from Gzip to ZStandard for ssate cache
files.
Zstandard compression provides a significant improvement in
decompression speed as well as improvement in compression speed and disk
usage over the 'tgz' format in use. Furthermore, its configurable
compression level offers a trade-off between time spent compressing
sstate cache files and disk space used by those files. The reduced disk
usage also contributes to saving network traffic for those sharing their
sstate cache with others.
Zstandard should therefore be a good choice when:
* disk space is at a premium
* network speed / resources are limited
* the CI server can sstate packages can be created at high compression
* less CPU on the build server should be used for sstate decompression
(From OE-Core rev: 0710e98f40ec7394567ede1f88f7c7bae306b925)
Signed-off-by: Henry Kleynhans <hkleynhans@fb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since Python 3.8 visit_Num(), visit_Str() and so on are all deprecated
and replaced with visit_Constant. We can't yet remove the deprecated
functions until we require 3.8, but we can implement visit_Constant to
silence the deprecation warnings.
(Bitbake rev: 4edd5767fc6d699f5262862b763b6a99ad1f1bbf)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
npmsw(and maybe other fetchers) may try to download same files simulatenously.
if a premirror is set, download method tries premirror location first. If file
is not found, done is set to false. In the meantime, other fetchers may download
the files required and verify_donestamp might return true.
In this scenerio, fetcher doesn't try to download again and fails. Do not
attempt to verify_donestamp if done is not set. By this way, we ensure that
fetcher attempts to do another download after premirror failure.
(Bitbake rev: 4fa25714916e84f99ecd22cb76cb5adada01e5e8)
Signed-off-by: Caner Altinbasak <cal@brightsign.biz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If multiple npmsw fetchers are trying to download the same npm file, one of them
can try to download the file while other is calling verify. npmsw methods gets
called without holding the lock, which causes race conditions in fetching and
verification etc. Lock the lockfile before calling proxy fetcher methods.
(Bitbake rev: fa39e6689d0f0fff772e1c81682698f4b1587b8a)
Signed-off-by: Caner Altinbasak <cal@brightsign.biz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since OE is about to change to zstd compression of sstate, it would make it
timely to convert the siginfo files from pickle which isn't reproducible
to json which is both reproducible and also human readable. At the same time
add zstd compression. This makes the siginfo files smaller, reprodubicle
and easier to debug.
Backwards compatibility mixing the two formats hasn't been supported since
in reality if sstate changes at the same time, files will be in one format
or the new one but comparing mixed formats won't make much sense.
Since json doesn't support sets, we translate them into lists in the files
themselves. We only use sets in bitbake since it makes things easier in
the internal code, sorted lists are fine for the file format.
[YOCTO #13973]
(Bitbake rev: 22c18494c9072788e6e26eb73de70378ae5c5bf5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The code works fine if we pass a file descriptor in and we need to
do this from the siggen code so add that as a valid input.
(Bitbake rev: fc8d74b7de576005ecf67920501dc7e694880fda)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For some debugging, BB_CURRENTTASK is set too late to be useful as
it isn't present in some event handlers for example. There is no
other way to know which task is actually running so set the value
earlier.
(Bitbake rev: 7daaaaa27f55b5a458656857c6d61a51b34a62fe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now reproducible builds are the default move the qemu-usermode check to anon
python code so that invalid configurations are detected.
(From OE-Core rev: 6c195acb400efeeebdacdd8d081b35f7417a694c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
sstate has been sufficiently invalidated since this change was made, drop
the workaround now.
(From OE-Core rev: 7a497d8f30f21bafc78d0f22f3442a9cc99544cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since Python 3.8 visit_Num(), visit_Str() and so on are all deprecated
and replaced with visit_Constant. We can't yet remove the deprecated
functions until we require 3.8, but we can implement visit_Constant to
silence the deprecation warnings.
(From OE-Core rev: 875a944392a3c93f40081a14af357f70b6b8264f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since Python 3.8 visit_Num(), visit_Str() and so on are all deprecated
and replaced with visit_Constant. We can't yet remove the deprecated
functions until we require 3.8, but we can implement visit_Constant to
silence the deprecation warnings.
(From OE-Core rev: abc93390a3f19bc4cc159c5690a478b9e2270906)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add the ability to replace DEPLOY_DIR_IMAGE with that path in
QB_OPT_APPEND. This allows for anything present in DEPLOY_DIR_IMAGE to
be passed into the qemu parameters. This is especially useful if you
want to run multiple flash images (as -bios only allows for one).
(From OE-Core rev: 1f0bec0421617e8aa9645c385195a755f0d44e75)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If gpg is used, it will find the first gpg agent in the path, this
may lead to issues where gpg comes from the host, and the agent
comes from a gnupg-native due to package signing. The versions
being out of sync causes gpg to fail.
(From OE-Core rev: 2b59fb9d52a405a32a1d069d4c5320b72fbd35ce)
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Bison's autoconf is also very good at finding textstyle, force it
to not find it unless it's explictly enabled.
(From OE-Core rev: 4f80931136462546f70f5e40d86d5e566f78d2e3)
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(From OE-Core rev: cff6888f3b2b4bd0a42329b7f7c59b33c9d51265)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Tasks shouldn't vary dependning on the value of BB_CURRENTTASK. They
happen not to due to when bitbake sets this but to fix other issues,
bitbake needs to set it earlier. Therefore exclude from hashes
globally.
(From OE-Core rev: 04a4ed1c509b0ef6af6d9ff53901c674ca402252)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Exceptions print more clearly using repr() instead of str(), fix
in fetch and unpack tasks.
Drop part of the test which no longer makes sense after this change.
(From OE-Core rev: c94816259cc1c09746353ad26ca0c811e0c962c2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This code pattern isn't very pythonic, improve it to drop the unneeded
len() calls.
(From OE-Core rev: 69376ac1a6147b26fe1abaa4cf68414024814d63)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(From OE-Core rev: 3b4ce5f69cd962cd219aa3c0e7a2de9096e2ecb8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The checkstatus semantics are unconventional, so document them.
(Bitbake rev: efb949020215b580e7dc3694c377b18df2fd7e9c)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
native and target 0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch
replaced by native-only 0001-Lib-sysconfig.py-use-prefix-value-from-build-configu.patch
which is more reboust against upstream changes, and keeps target code unmodified.
This however necessitated adding 0001-sysconfig.py-use-platlibdir-also-for-purelib.patch
to avoid hardcoding 'lib' on target builds as libdir.
Drop chunk from 0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch as
upstream now uses sysconfig directly inside distutils.
Add 0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch and
0001-multiprocessing-disable-a-failing-test.patch to address ptest failures.
License-Update: copyright years, case corrections.
(From OE-Core rev: 72a75043a946f7db01d3ec04c8889e055f542cca)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When DEBUG_BUILD is set for building rust-native, it generated
error as follows:-
=========================================================
Building : rustdoc, rustdoc-json-types
error[E0463]: can't find crate for `rustc_llvm`
which `rustc_driver` depends on
--> src/librustdoc/lib.rs:37:1
|
37 | extern crate rustc_driver;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
=========================================================
On analyzing the issue, it was found that rustc_llvm crate was present
at required path. However, it was very huge due to the presence of
debugging information. Hence, it was somehow not recognized as a valid
crate. The following patch removes the debug information from
rust-llvm-native which is built prior to rust-native but retains debug
information as required in rust-native binaries.
(From OE-Core rev: 7261a4b4d5778a48c8d72c9125233b1b0bc009e6)
Signed-off-by: Pgowda <pgowda.cve@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are situations when the user have the 'patchdir' defined
as a parameter on SRC_URI. However he doesn't know that with this
the patch is applied relatively to the receipe source dir 'S'.
- When user have 'patchdir' defined check if this directory exist.
- If the patch fails show addition info to the user:
- Import: show the striplevel
- Resolver: show the expanded 'patchdir' to the user.
The next example is from opencv in meta-oe layer, here the
patch is applied on the target directory ${WORKDIR}/git/contrib.
S = "${WORKDIR}/git"
SRCREV_FORMAT = "opencv_contrib"
SRC_URI = "git://github.com/opencv/opencv.git;name=opencv \
git://github.com/opencv/opencv_contrib.git;destsuffix=contrib;name=contrib \
file://0001-sfm-link-with-Glog_LIBS.patch;patchdir=../contrib \
"
* When the patch fail there are no message that indicates the real reason.
patchdir=../no-found-on-file-system
ERROR: opencv-4.5.2-r0 do_patch: Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch
can't find file to patch at input line 37
Perhaps you used the wrong -p or --strip option?
* The check of the patchdir will add a new fatal error
when the user specifies a wrong path than don't exist.
patchdir=../no-found-on-file-system
ERROR: opencv-4.5.2-r0 do_patch: Target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../no-found-on-file-system' not found, patchdir '../no-found-on-file-system' is incorrect in patch file '0001-sfm-link-with-Glog_LIBS.patch'
* When we can't aplly the patch but the patchdir exist,
show the expanded patchdir on fatal error.
patchdir=../git
ERROR: opencv-4.5.2-r0 do_patch: Applying patch '0001-sfm-link-with-Glog_LIBS.patch' on target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../git'
Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch
can't find file to patch at input line 37
Perhaps you used the wrong -p or --strip option?
(From OE-Core rev: c44bc7c0fb8b7c2e44dd93607a3bfd9733e1df80)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(From OE-Core rev: 71780bb9ac3c9e096888128d9cf3441c1758d816)
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Drop pluggy-1.0 patch; fixed upstream
pytest 6.2.5 (2021-08-29)
Trivial/Internal Changes
#8494: Python 3.10 is now supported.
#9040: Enable compatibility with pluggy 1.0 or later.
References:
https://github.com/pytest-dev/pytest/pull/8494
https://github.com/pytest-dev/pytest/pull/9040
(From OE-Core rev: cd83fa054d133eb2b189ad3f0a743cc72001e064)
Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Mike Crowe
Khem Raj
Daniel McGregor
Michael Halstead
Henry Kleynhans
Ross Burton
Caner Altinbasak
Jon Mason
Alexander Kanavin
Pgowda
Jose Quaresma
Zoltán Böszörményi
Tim Orling