summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2023-10-09Document BB_HASHSERVE_MAX_PARALLELjpew/bitbake-hashserver-wsJoshua Watt
Adds documentation to describe how BB_HASHSERVE_MAX_PARALLEL should be used Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09runqueue: Query unihashes in parallelJoshua Watt
Instead of serially querying unihashes, marshal up all of the tasks that have satisfied dependencies and query the unihashes for them all at once (potentially in parallel). Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09sstatesig: Implement new siggen APIJoshua Watt
Implements the new API required for querying unihashes in parallel Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09siggen: Add parallel query APIJoshua Watt
Implements a new API called get_unihashes() that allows for querying multiple unihashes in parallel. The API is also reworked to make it easier for derived classes to interface with the new API in a consistent manner. Instead of overriding get_unihash() to add custom handling for local hash calculating (e.g. caches) derived classes should now override get_cached_unihash(), and return the local unihash or None if there isn't one. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Add Client PoolJoshua Watt
Implements a Client Pool derived from the AsyncRPC client pool that allows querying for multiple equivalent hashes in parallel Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09asyncrpc: Add Client Pool objectJoshua Watt
Adds an abstract base class that can be used to implement a pool of client connections. The class implements a thread that runs an async event loop, and allows derived classes to schedule work on the loop and wait for the work to be finished. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Implement read-only version of "report" RPCJoshua Watt
When the hash equivalence server is in read-only mode, it should still return a unihash for a given "report" call if there is one. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09Ensure PR server connections are closedJoshua Watt
Ensures that connections the PR server are properly closed to eliminate warnings about unclosed event loops Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09contrib: hashserv: Add docker-composeJoshua Watt
Adds a docker-compose.yaml file and docker-compose script wrapper to run it with the correct arguments. This can be used to easily standup a hash equivalence server with a postgresql database for demonstration purposes. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09contrib: Update hashserv DockerfileJoshua Watt
Updates the hash equivalence docker file to the latest Alpine version, and also to install the required python modules into a virtual environment Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Add SQLalchemy backendJoshua Watt
Adds an SQLAlchemy backend to the server. While this database backend is slower than the more direct sqlite backend, it easily supports just about any SQL server, which is useful for large scale deployments. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Abstract databaseJoshua Watt
Abstracts the way the database backend is accessed by the hash equivalence server to make it possible to use other backends Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09bitbake-hashserv: Allow arguments from environmentJoshua Watt
Allows the arguments to the bitbake-hashserv command to be specified in environment variables. This is a very common idiom when running services in containers as it allows the arguments to be specified from different sources as desired by the service administrator Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09asyncrpc: Prefix log messages with client infoJoshua Watt
Adds a logging adaptor to the asyncrpc clients that prefixes log messages with the client remote address to aid in debugging Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: tests: Add external database testsJoshua Watt
Adds support for running the hash equivalence test suite against an external hash equivalence implementation. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09asyncrpc: Add context manager APIJoshua Watt
Adds context manager API for the asyncrcp client class which allow writing code that will automatically close the connection like so: with hashserv.create_client(address) as client: ... Rework the bitbake-hashclient tool and PR server to use this new API to fix warnings about unclosed event loops when exiting Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Add websocket connection implementationJoshua Watt
Adds support to the hash equivalence client and server to communicate over websockets. Since websockets are message orientated instead of stream orientated, and new connection class is needed to handle them. Note that websocket support does require the 3rd party websockets python module be installed on the host, but it should not be required unless websockets are actually being used. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09asyncrpc: Abstract socketsJoshua Watt
Rewrites the asyncrpc client and server code to make it possible to have other transport backends that are not stream based (e.g. websockets which are message based). The connection handling classes are now shared between both the client and server to make it easier to implement new transport mechanisms Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Rename clean_unused -> clean-unusedJoshua Watt
Because it will bother me that it doesn't match the others Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09bitbake-hashclient: Add clean-unused subcommandJoshua Watt
Adds a subcommand to clean unused outhash entries from the server based on age Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Add API to clean unused entriesJoshua Watt
Adds an API to remove unused entries in the outhash database based on age and if they are referenced by any unihash
2023-10-09hashserv: Extend get_outhash API to optionally include unihashJoshua Watt
Extends the get_outhash API with a flag indicating whether to include the unihash in the output. This is means that the query doesn't require the unihash entry to be present to return a result Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09bitbake-hashclient: Add remove subcommandJoshua Watt
Adds a subcommand to invoke the remove API on the server [YOCTO #15064] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-09hashserv: Add remove APIJoshua Watt
Adds a `remove` API to the client and server that can be used to remove hash equivalence entries that match a particular critera Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
2023-10-08devtool/upgrade: check all git config locationsMarcus Flyckt
Fixes [YOCTO #15222] In some situations its more practical to keep git configuration at `/etc/gitconfig` instead of `$HOME/.gitconfig` (e.g., when mounting git configuration into a docker container). This change makes `devtool upgrade` consider any available git configuration instead of only checking `--global`. (From OE-Core rev: 30a9f7de45050c8bac49d4b37419cc2e067a75fa) Signed-off-by: Marcus Flyckt <marcus.flyckt@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-08qemurunner.py: detect login prompt without without utf-8 conversionMikko Rapeli
In case utf-8 conversion of the serial console data is flaky due to bad characters in the stream. (From OE-Core rev: 3f4921ced1e2aae546ff7cef232eec3e214c28be) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06busybox: Set PATH in syslog initscriptMartijn de Gouw
This script is not always called with /sbin and /usr/sbin in the PATH already, for example when called via ssh. Explicitly set PATH to make sure it includes /sbin and /usr/sbin since that's where start-stop-daemon is located. (From OE-Core rev: fa53f898eaba15dff030f9eadf86e5bca7d954fa) Signed-off-by: Martijn de Gouw <martijn.de.gouw@prodrive-technologies.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06qemu: fix CVE-2023-42467Yogita Urade
QEMU through 8.0.0 could trigger a division by zero in scsi_disk_reset in hw/scsi/scsi-disk.c because scsi_disk_emulate_mode_select does not prevent s->qdev.blocksize from being 256. This stops QEMU and the guest immediately. References: https://nvd.nist.gov/vuln/detail/CVE-2023-42467 https://gitlab.com/qemu-project/qemu/-/issues/1813 (From OE-Core rev: 7c42b976d7a72acf917bae9d055768a1350e507d) Signed-off-by: Yogita Urade <yogita.urade@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06linux-firmware: create separate package for cirrus and cnm firmwaresFahad Arslan
Some licenses only allow usage of corresponding firmwares when a specific hardware is present. This requires split of such firmwares from linux-firmware package to firmware specific sub package. As this split is based off of licensing, it makes sense to group firmware blobs having the same license in the same package. This commit is a first step in this direction, and creates separate packages for cirrus and cnm firmware. (From OE-Core rev: 53d9d8789efc701609a5a1e985287344c2209d62) Signed-off-by: Fahad Arslan <fahad.arslan@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06gstreamer: upgrade 1.22.5 -> 1.22.6Archana Polampalli
This release only contains bugfixes and security fixes. Highlighted bugfixes in 1.22.6: Security fixes for the MXF demuxer and H.265 video parser Fix latency regression in H.264 hardware decoder base class androidmedia: fix HEVC codec profile registration and fix coded_data handling decodebin3: fix switching from a raw stream to an encoded stream gst-inspect: prettier and more correct signal and action signals printing rtmp2: Allow NULL flash version, omitting the field, for better RTMP server compatibility rtspsrc: better compatibility with buggy RTSP servers that don't set a clock-rate rtpjitterbuffer: fix integer overflow that led to more packets being declared lost than have been lost v4l2: fix video encoding regression on RPi and fix support for left and top padding waylandsink: Crop surfaces to their display width height cerbero: recognise Manjaro; add Rust support for MSVC ARM64; cmake detection fixes various bug fixes, build fixes, memory leak fixes, and other stability and reliability improvements https://nvd.nist.gov/vuln/detail/CVE-2023-40475 https://nvd.nist.gov/vuln/detail/CVE-2023-40476 https://gstreamer.freedesktop.org/releases/1.22/#1.22.6 (From OE-Core rev: ff2fc789e08c42903f5e528d4836dd482227291c) Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06libtirpc: conditionally enable gssapiDaniel McGregor
This is for upcoming work to support gssapi in nfs-utils for nfsv4 and kerberos mountpoints. (From OE-Core rev: ae9c64cd2291f4b24ac442816a4e354e47c70933) Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06meson: upgrade 1.2.1 -> 1.2.2Daniel McGregor
The newly released version 1.2.2 adds version 17 to CMake LLVM detection, so we will now choose the OE provided LLVM version instead of the one on host during native builds. (From OE-Core rev: 90edba29f9f4c115af5202c566eaa935656de1eb) Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06recipetool/create_buildsys_python: use importlib instead of impChris Laplante
'imp' was deprecated in Python 3.4 and removed in 3.12. The piece of importlib we use has been around since 3.3. (From OE-Core rev: 457f0dad87b4e45a53865b5ad2c150215bd74019) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06packages.bbclass: Correct the check for conflicts with renamed packagesPeter Kjellerstedt
The original solution replaced all overrides with the package name that was being checked. This can have unforseen consequences where some variable involved in defining the value for the PKG:<package> variable may rely on some override which is not set as expected. It also meant that any PKG variable set using an override would not be caught, e.g., PKG:${PN}:${MACHINE} = "${PN}-dev" (made up example that would always fail with the old code). (From OE-Core rev: de62d538dbfe6caf123ff366643f893077175583) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06scripts/oe-setup-layers: Update how to determine if directory is git repoJermain Horsman
Previously _is_repo_git_repo() would return a result containing b'true\n' or b'false\n' if 'git rev-parse' ran successfully, instead of True of False. While this can be solved using e.g. result.strip().decode("utf-8") == "true", there are some other cases to consider. First, .git can be a file and not a directory when using a worktree. Second, an emtpy .git directory in 'repodir' for which some parent of 'repodir' is an actual git repo will still return True in this case. To account for these cases as well, use 'git rev-parse --show-toplevel' and compare the result against 'repodir' instead of using 'git rev-parse --is-inside-git-dir'. (From OE-Core rev: 0830f53900dd7fd669a7d6492325559ad3225302) Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06autotools.bbclass: Add *.m4 to CONFIGURE_FILESPeter Kjellerstedt
This makes do_configure depend on changes to any m4 file when externalsrc is active. (From OE-Core rev: 492559191943c9e2666c3dda1824c5aafbe487d5) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06externalsrc.bbclass: Support specifying patterns in CONFIGURE_FILESPeter Kjellerstedt
This allows, e.g., *.cmake to be added to CONFIGURE_FILES to make the do_configure task depend on changes to any cmake file. (From OE-Core rev: 09873b3fb24a00cfbd73282d29e4c5821774f579) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06bitbake: selftest/fetch: Ensure top level directory timestamp doesn't break testRichard Purdie
The infrastructure changes highlighed an issue where the top level directory mtime stamp isn't included in the tarballs. This isn't an issue the test is meant to check for so ignore it. (Bitbake rev: 5a1e5b8dc55d6bb4d93bac3492f8c43ff957e712) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06bitbake: toaster: Monitoring - implement Django logging systemAlassane Yattara
(Bitbake rev: 2efb146480ee46c0463d9edb71bf1c03ce15bcf2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06bitbake: toaster: update selenium version and code syntaxMarlon Rodriguez Garcia
Updated selenium version to latest 4.13.0, changed selenum specific version syntax elements to accomplish test success (Bitbake rev: 868c88a249ef4b9fe5a891e76e25e054e4fcd994) Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-05libxpm: upgrade to 3.5.17Ross Burton
This release fixes the following CVEs: - CVE-2023-43788 - CVE-2023-43789 (From OE-Core rev: 46dd8ce41756dbc2aa0f9001416f208cced1c8d5) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-05libx11: upgrade to 1.8.7Ross Burton
This incorporates fixes for the following CVEs: - CVE-2023-43785 - CVE-2023-43786 - CVE-2023-43787 (From OE-Core rev: a1534bb34b680bfc5cb2f35b5fd5a0c2afed6368) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-05uboot-extlinux-config.bbclass: fix missed override syntax migrationQuentin Schulz
uboot-extlinux-config allows to specify multiple "labels" (entries in a menu, à-la grub) and each of them have their own values for some fields. Each "base" variable, e.g. UBOOT_EXTLINUX_FDT can be overridden for each label. This is done via the OVERRIDES mechanism based on the label name, e.g. UBOOT_EXTLINUX_FDT:linux if linux is a label. However, OVERRIDES doesn't contain the label globally because it's only necessary in one task. Therefore, the OVERRIDES itself is modified within the task. This means that the sigdata will not be told the dependency on UBOOT_EXTLINUX_FDT:linux, because it cannot know about it. For this reason, we need to explicitly specify which variables this task depends on via vardeps varflag for the task. This was done in the past, but we missed updating it during the override syntax migration so the cache wouldn't get invalidated if someone modifies UBOOT_EXTLINUX_FDT:linux from a configuration file or a bbappend for example. Let's fix this by migrating it to the new syntax. (From OE-Core rev: b4dd9d873508128adbbf5ff6cf0a3df3d2ffbcf6) Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-05oeqa/concurrencytest: Remove invalid buffering optionRichard Purdie
Fix warnings from oe-selftest -j: /usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used return io.open(fd, mode, buffering, encoding, *args, **kwargs) Remove the option since it clearly doesn't do much. (From OE-Core rev: 6b872ee72942951fd464c4c6cb9eadcb9b4749c1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-05linux-yocto/6.5: integrate fixes for sanity issuesBruce Ashfield
making the following commits available in our 6.5 kernel: 49e3d8448aea locking/atomic: scripts: fix fallback ifdeffery 9ba8e064374d crypto: jitter - add RCT/APT support for different OSRs 50f59f46583a crypto: jitter - Add clarifying comments to Jitter Entropy RCT cutoff values Which should address several of the failures we are seeing with 6.5 under integration testing. (From OE-Core rev: dd8f47a9c4407741575e9264f7c44e4623f606e0) Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04u-boot: Upgrade to 2023.10Fabio Estevam
Upgrade to U-Boot 2023.10. The U-Boot 2023.10 tag is in the master branch, so switch the branch back to master. (From OE-Core rev: 6fce7f99ff5bacf7ad4b1816f462260fe41b38ee) Signed-off-by: Fabio Estevam <festevam@denx.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04uboot-extlinux-config.bbclass: Add menu title configurationDaniel Semkowicz
Add new UBOOT_EXTLINUX_MENU_TITLE variable that allows configuring the "MENU TITLE" entry. If set to empty, "MENU TITLE" will not be added to the output file. (From OE-Core rev: 23026911142585fde9290e21b07934fc583b6540) Signed-off-by: Daniel Semkowicz <dse@thaumatec.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04uboot-extlinux-config.bbclass: Uppercase "menu title" entryDaniel Semkowicz
All other extlinux.conf entries are written to the output file in uppercase. (From OE-Core rev: 6c89654cf37da95aeea07e1645f2cdffe320c8bc) Signed-off-by: Daniel Semkowicz <dse@thaumatec.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04uboot-extlinux-config.bbclass: Remove repeated space characterDaniel Semkowicz
(From OE-Core rev: e8a7a8d93c20f966bc2845d23696d01234cbb227) Signed-off-by: Daniel Semkowicz <dse@thaumatec.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04python3-numpy: remove obsolete reproducible workaroundRoss Burton
We currently delete some pycache files because frozenset wasn't able to be reproducible, but this has been fixed in Python 3.11: https://github.com/python/cpython/commit/33d95c6facdfda3c8c0feffa7a99184e4abc2f63 (From OE-Core rev: 5c287a896fa9f5cd05b6a2411528fdc8fb2579d6) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-20 04:06:19 +0000