| Age | Commit message (Collapse) | Author |
|---|
|
Remove various build host references from packages:
libstdc++-staticdev
gcc-runtime.dbg
The references are removoved by correctly setting various compiler
-fdebug-prefix-map settings. There are two main issues:
The default DEBUG_PREFIX_MAP variable references WORKDIR, however,
gcc sources are in a shared folder (work-shared)/
Additionally, DWARF info seems to store symlink names but gcc
seems to resolve symlink names referenced in -fdebug-prefix-map.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Use "pigz -n -T" to avoid timestamps from compressed file headers.
Using "gzip -n" may not work if gzip is actually pigz in disguise.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
pigz is not always a direct drop-in replacement for gzip.
"pigz -n" is not equivalent to "gzip -n"
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
By using the full mrproper clean target it also removes some generated files
that are not required on the target to build. It also helps to fix reprodudible
builds as the generated files could have different permissions causing the
check to fail.
Tested with building and booting kernel from source on the target
[YOCTO #12137]
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Building various libraries (libc6, libc6-pic, libc6-staticdev, libc6-dbg, ...)
can be non-deterministic because they may be built with two different versions
of intl/plural.c. in two otherwise identical builds. We may or may not re-generate
the file plural.c from the file plural.y, based on bison being installed or not
and based on mtimes of those two files, as the Makefile contains:
plural.c: plural.y
$(BISON) $(BISONFLAGS) $@ $^
If the above rule does not fire, we use a "fallback" plural.c, otherwise
we use plural.c re-generated from plural.y.
The fix is to always require bison to be installed and unconditionally
re-generate plural.c. (This is achieved by simply removing the "fallback"
plural.c from the distribution).
The generated plural.c can contain build host references in the form of
"#line" statements. These contain absolute build host paths, so we use
bison "--no-lines" additional option when generating plural.c.
[YOCTO #12291]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
1. Remove build host references from CFLAGS in generated config.h file.
They end up compiled in the image, but are are only used for information.
2. Remove PA_BUILDDIR hardcoded references: safe to remove as we are guaranteed
not to run from the build directory when the image is cross-compiled
3. Take care of any embedded __FILE__ strings, only if the compiler supports
-ffile-prefix-map option.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove remaining build host references from packaged files.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
ltp package contains several gzipped files.
Improve reproducibility of the build by ensuring the gzipped files
do not contain timestamps in their headers.
https://wiki.debian.org/ReproducibleBuilds/TimestampsInGzipHeaders
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Some examples used "random" numbers to generate random RGB colors.
This would break reproducible builds.
The issue was solved by a patch backported from Debian.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Patch backported from Debian:
https://sources.debian.net/src/groff/1.22.3-9/debian/patches/source-date-epoch.patch/
Various documents contain information such as:
%%Creator:·groff·version·1.22.3
%%CreationDate:·Sat·Nov·11·01:04:26·2017
The intent is to replace CreationDate by a value specified
by SOURCE_DATE_EPOCH (if present in environment).
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove several build host references from modinfo.sh files.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove build host refeences.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Improve reproducibility: Remove all build host references from
distributed files.
Do not package non-linux OS related files.
Also remove some additional files not needed by run-ptest. (There are
probably still more files that can be removed, but as long as they
don't leak build host references they are harmless).
Fix the expected checksum of MakeMaker/lib/ExtUtils/Liblist/Kid.pm
to match the one expected: We modified the file, but did not
recalculate/update the file checksum accordingly.
(This fixes the only failing test.)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove remaining build host references from packaged files.
[#YOCTO 11472]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove various build host references scattered within
comments in numerous files that are distributed in the package.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Make sure build host references do not end up being compiled in the image.
This only affects libicutu and icu-dbg.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove additional build host references from Config_heavy.pl
and config.sh.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove build host references from the file LibIcalTargets-noconfig.cmake.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Sanitize generated config.h containing build host reference such as:
#define POSIX_SHELL "<builddir>/tmp/hosttools/bash"
The path for POSIX_SHELL is not only wrong for the cross-compiled target,
it also prevents reproducible build.
Prior the compilation we edit the value to:
#define POSIX_SHELL "bash"
With this change we can build reproducible libgnutls-bin, libgnutls-dev
and libgnutls-dbg packages.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Delete various build host references from the internally
generated file sudo_usage.h. The references get compiled into
executables, which leads to non-reproducible builds.
The removed references (configure options) were only used as part
of the sudo "usage", and even then only when ran as root.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Improve reproducible build of curl-dev and curl-dbg packages.
curl-dev: Correctly remove build host references from curl-config
curl-dbg: Do not generate time stamps in files generated by mkhelp.pl
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Improve reproducible build of:
openssl-staticdev
openssl-ptest
openssl-dbg
libcrypto
There are three main causes that prevent reproducible build:
1. openssl-ptest: Remove build host references from Makefile and Configure distributed
with the package
2. openssl-dbg: Generate modified "buildinf.h".
This file is generated by the script "mkbuildinf.pl". The generated file contains
build host CFLAGS, containing various build host references. We need to pass sanitized
CFLAGS to the script.
3. We also need to modify the script "mkbuildinf.pl" itsel in order not to generate
a timestamp if SOURCE_DATE_EPOCH is present in the environment.
With the newly generated "buildinf.h" we can build reproducible libcrypto and
openssl-staticdev as well.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove several Makefiles containing build host references.
While at it, also remove some additional files not needed for
testing.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove all build host references from several distributed files.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
The wrappers are needed for cross-compiling only.
[YOCTO #11705]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
The tclConfig.sh is also used by other packages (such as expect) for
cross-compiling, the host path from it can't be removed directly in
the do_install step.
With PACKAGE_PREPROCESS_FUNCS to remove host path and avoid the
crossscripts installed to target.
Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
|
|
On occassions two builds of all glibc-gconv packages could differ in:
build1: Depends:·glibc-gconv
build2: Depends:·glibc-gconv,·libc6·(>=·2.25)
This patch attempts to fix this.
[YOCTO #11551]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Use gzip compression without timestamps in the metadata.
(Use gzip -n).
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
The original code only enabled a sub-set of all available tests.
It also copied binary images although the images were already installed
in a different location as well. In addition, the original
code copied libtool scripts instead of already cross-comiled binary images.
This patch modifies the test scripts so there is no need to copy
images already installed: instead it tests images already installed.
The images are scattered in /bin, usr/bin, /sbin/ usr/sbin folders,
so the modification of test scripts reflects this. We do, however,
copy some cross-compiled tests that were missing previously.
By the virtue of not copying the libtools scripts we also managed
the achieve binary reproducible package, as previously leaked build host
info was contained in libtool scripts, which are not copied anymore.
[YOCTO #10953]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
"buildhistory_list_files" generates text files containing list of files
that go into various packages/images. The file list contains file sizes as
well. However, the listed "file size" of a directory entry depends
on the file system of the build host. So, for example, building on EXT4 the
folder size is always listed as 4096 bytes. The same folder is listed with different
size when building on tmpfs. This can make direct comparison of two build histories
difficult.
Considering that the listed folder size is meaningless anyway (it has no relation to
the size of the package/image), this patch addresses the problem by removing
the directory entry sizes entirely from the generated text files.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove build host references
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Compress disctcc-doc without timestamps in the meta-data header.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Remove several build host references from modinfo.sh files.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Simplify building reproducible images by using
DISTRO="poky-reproducible"
Sets some variables to reasonable values so users do not
have to set them in local.conf.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Conditionally support binary reproducibility of rootfs images.
If BUILD_REPRODUCIBLE_BINARIES = 1 then:
1. set /etc/timestamp to a reproducible value
2. set /etc/version to a reproducible value
3. set /etc/gconf: set mtime in all %gconf.xml to reproducible values
The reproducible value is taken from the variable REPRODUCIBLE_TIMESTAMP_ROOTFS.
If the variable is not specified, the timestamp value is derived from
the top git commit.
[YOCTO#11176]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
By default, bitbake uses host cpio, which can be quite old
and missing crucial newever features.
This patch allows to use the cpio-native instead, which is the
latest upstream version. To use the cpio-native instead of the
cpio from host, you need to specify:
do_image_cpio[depends] += "cpio-replacement-native:do_populate_sysroot"
EXTRANATIVEPATH += "cpio-native"
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
This patch helps to build cpio images that are binary reproducible.
The changes are as follows:
1. By default, cpio from the host is used, which can be quite old.
Hence we need to implement a way to use/call cpio-native, which supports
new features needed for binary reproducibility, notably the arguments
such as "--reproducible" and "--ignore-devno".
This can be achieved by specifying the following (in local.conf):
do_image_cpio[depends] += "cpio-replacement-native:do_populate_sysroot"
EXTRANATIVEPATH += "cpio-native"
2. Provide a way to pass custom arguments to cpio. This is done via
a new variable IMAGE_CMD_CPIO. For binary reproducible cpio archives
one needs to set (in local.conf)
IMAGE_CMD_CPIO = "cpio --ignore-devno --reproducible "
3. A symlink is created as part of the image. Here we make sure it gets the
timestamps (mtime) based on the timestamp of the symlink target, rather than
the timestamp corresponding to the build time.
4. Sort the files we pipe to cpio.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Use gcc option "-ffile-prefix-map" if available.
This will remap any hard-coded __FILE__ containing build host
path to a reproducible value, with the absolute build-host path
stripped.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
In order to achieve binary identical modules-${MACHINE}.tgz, we need to
instruct "tar" to set mtime for added files based on SOURCE_DATE_EPOCH
when creating the file. We also need to instruct "gzip" not to save the
original file name and time stamp in metadata.
Other minor code improvements without functional changes.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
As we may be using external toolchains, we cannot assume
various compiler options to be valid.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
Conditionally set some environment variables in order to achieve
improved binary reproducibility. Providing BUILD_REPRODUCIBLE_BINARIES is
set to "1", we set the following environment variables:
export PYTHONHASHSEED=0
export PERL_HASH_SEED=0
export TZ="UTC"
Additionally, we export and set SOURCE_DATE_EPOCH. This is the most crucial step to
achieve binary reproducibility. The value for this variable (timestamp) is
obtained after source code for a recipe has been unpacked, but before it is patched.
If the code sources come from a GIT repo, we get the timestamp from the top
commit. (GIT repo does not preserve file mktime timestamps). It is not safe to assume
folders named "git" contain git repositories, so we check for presence of .git folder
in ${S}. Otherwise, if GIT repo is not present, we get mtime from known files such as
NEWS, ChangeLog, etc. If this also fails, we go through all files and get the timestamp
from the youngest one. We create an individual timestamp for each recipe.
The timestamp is stored in the file '__source_date_epoch.txt' located in ${WORKDIR}.
Later on, each task reads this file and sets SOURCE_DATE_EPOCH based on the value found
in the file.
The file __source_date_epoch.txt file is re-used if found. This can be the file
we previously created ourselves, or it can be a file provided by a user via
a recipe.
[YOCTO#11178]
[YOCTO#11179]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
|
|
The newly added file in the last commit was corrupted, fix it.
(Bitbake rev: be393f247a08c0a4a50a6a76b8fd57f78295d2a1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove "prettify.js" and "prettify.css" due to license issues with Apache2.
Replace with "highlight.pack.js" with its BSD3 License.
[YOCTO #12206]
(Bitbake rev: 6361698819530382541506b06a61f2c76dde59cb)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Brian Avery <brian.avery@intel.com>
Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I have been working with OE on debian-9 for several months now without
issue. In addition, I tested a build + runqemu for core-image-sato
following the quickstart guide and had no issues.
(From meta-yocto rev: 2d8a572df7498ce8eb3a78f05384f0f7dd5ecb91)
Signed-off-by: Martin Kelly <mkelly@xevo.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Force items into packagegroup-core-tools-profile? I don't think so.
(From meta-yocto rev: df0f331832fad818604d8696700765fb4d3ba072)
Signed-off-by: Joe Slater <jslater@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add tools required by testimage to HOSTTOOLS only when testimage is
inherited. These tools, as described in the comment, are only required
by the testimage task. So this change should not have negtive effect.
This would also solve build error on hosts which miss some tool such as scp.
(From OE-Core rev: 8ca61a5464743ff85b6d26886112750d6ddd13e0)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes
internal compiler error: Max. number of generated reload insns per insn is achieved (90)
(From OE-Core rev: d2631f45a057c53797b7ba657662f35f66a2b04e)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
e2fsprog testsuite used to log results into its own log file into
/usr/lib/e2fsprogs/ptest/test.log. Therefore console output was not available and
redirecting ptest-runner output into a log file would not capture output of any
individual tests. So overall the whole e2fsprogs testsuite consisting of about
300 tests was evaluated as a single test.
This patch ensures the e2fsprogs test_script output is not automatically redirected
while executing during run-ptest. Any redirection is up to the user.
Additionally, the results of tests are prefixed by more canonical "PASS:" FAIL:"
based on the actual test results.
Aditionally, remove various files created by the testsuite.
[YOCTO #12146]
(From OE-Core rev: a218b85dba1a481901548f7414676c0ae6f43316)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This splits out the i.MX SDMA firmwares for i.MX6 and i.MX7 SoCs. This
also includes the required runtime provides, conflicts and replaces
for the old firmware-imx which was provided by NXP BSP layer.
(From OE-Core rev: b3f3078fd4349fdf6986dd57e4b04bce03630924)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Includes brief description of the project, pointers to website, documentation,
mailing list and source code.
(Bitbake rev: 28249c42701f9156a0b3153d72d7e46dacab37cb)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Juro Bystricky
Saul Wold
Wenzong Fan
Richard Purdie
David Reyna
Martin Kelly
Slater, Joseph
Chen Qi
Khem Raj
Otavio Salvador
Leonardo Sandoval