| Age | Commit message (Collapse) | Author |
|---|
|
In a system where the kernel source and object directories are
split we want the generated .config to be in the build directory
and not the current directory. The easiest way to support this is
to respect the O= environment variable.
Neither KBUILD_OUTPUT or O were propagating automatically to the
kernel build, so always explicitly setting O= and passing it to
the kernel build ensures that it is respected. Without this .config
always ends up in the current working directory.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Some environments export variables to subshells and in those
environments setting 'MAKE=true' will interfere with the kernel
build system's use of that same variable. To avoid this, we can
simply rename MAKE to MAKE_FLAG.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When debugging options that are output as not being in the final
.config file it is often useful to look at the fully concatenated
configuration file. To support this we add a debug flag that will
not cleanup the .tmp.config* file that is created. The risk of
having uncleaned .tmp.config* file is an acceptable tradeoff for
this debug assist.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
syncing the kern-tools variant of merge_config.sh to the upstream
kernel version, commit: 409f117e2d.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kconf_check is called from many different types of BSPs and
configuration architectures. As such, not all of the categories of
configuration groups will always be present (i.e. no hardware frags,
no required frags, etc).
To ensure that the varous cat, grep and sort lines to not try and
access non-existent files, we add extra checks, or touches to
files to ensure a consistent set of files before the various checking
phases run.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kernel branches are constructed during patching of the tree by
constructing a '/' based hierarchy of names as each branch
directive is encountered.
But if a "branch $name $branchpoint" is used, the entire branch
name is supplied so no additions to the hierarchy should
happen. As such, that type of branch command should not be part
of branch name calculation and preparation.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To make the generated feature file more useful to external scripts
that wish to offer a list of optional features supported by a kernel,
extra divisions are required in the kernel features rc file.
With this change, there are three categories:
- feature: features that are built into the kernel (not selectable)
- feature-optional: optional (and selectable) features in the kernel
- config: configuration features supported by the kernel
They can be queried like this:
% git config -f /tmp/kern-features.rc --get-all kernel-options.feature-optional
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When a series of patches fail to apply the pending patches are dumped
to the user. But if the GUILT_BASE directory is not set, you simply
get something that looks like an internal error:
| To force apply this patch, use 'guilt push -f'
| [ERROR] unable to complete push
| pending patches are:
| Patches directory doesn't exist, try guilt-init
If we export GUILT_BASE, the output is much better:
| [ERROR] unable to complete push
| pending patches are:
links/files/0002-makefile-patch.patch
| ERROR. Could not update git tree
| ERROR. Could not apply updates for qemux86.
| Patch failures can be resolved in the devshell:
| % bitbake -c devshell linux-yocto
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When forcing a branch switch to the desired branch we should be
using the proper 'branch' command. Since without this call, the
proper variables will not be set, and patches can't be applied
to the tree.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In the case of a custom repository, we may not enounter ANY explicit
branch points. This means that the first patch push will fail, since
the series file and other infrastructure has not been set up. To
ensure that we can always push patches, the starting branch is now
prepped for modifications.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When an explicit branch point is specified:
branch $new_branch $branch_point
it means that the name of the new branch is explicit (and
known), and the branch name shouldn't inherit the names of
the branches that have proceeded it.
So if this is detected, branch name generation is skipped.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The warning:
warning: could not find (or was already included):
Sometimes is not clear because the include path is omitted in scc. So
print the current feature directory as well.
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
|
|
Add the ability to specify a "keep" config file. This gives the
ability to force an option to be kept around, even if it violates
the cleaning rules.
From the help text:
An input config file with a list of configurations that should
be maintained regardless of other cleaning activities. These
are typically config options that should be declared as
'hardware' for the given board, and the keep entry would no
longer be necessary.
Note: This file is normally called 'hardware.cfg', so it can be
reused by the build process in future iterations.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
This is a partial revert of commit 5b6498c4bc02a
buildall: remove all instances of it using/reading scc files
The above was done, since the final branch used by an scc
file is not obvious (imagine scc A that includes scc B which
in turn does the actual branch selection). Since the output
of scc (the meta-series) clearly shows the final branch, the
switch over seemed obvious.
In the context of building all BSPs from a freshly generated
tree, we know the meta series files match what is on all the
branches. However if someone wants to use buildall on a
tree that is in maintenance mode, there may have been new
scc added (i.e. no meta), or configuration fragments relocated
(meta with stale paths) resulting in odd failures that would
not happen if the branch was built via the complete build
system.
This puts things back to being driven from the scc leaf nodes.
The stale meta problem is handled by regenerating them. There
is an optional "use what is there" knob that can be used by
people generating a fresh tree, since they know that there is
no risk of staleness in that one particular use case (and since
regeneration of them can be timewise expensive).
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Not all features will create branches with the same origin, to support
these features, we need to do several things:
- condition the branch name to be the first argument in "branch <foo> <bar>"
since <foo> is the name and <bar> is the branch point
- during tree generation the branch should be reset to a known starting
point between features.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
script.
Since the only current call to "createme" from do_patch() doesn't pass
a defconfig argument, remove the apparently superfluous references to
defconfig from createme, but leave the check_defconfig() function in
just in case it ever becomes useful.
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Fixing a long standing typo in a test for the verify_branch, which
was "verify_branch" vs "$verify_branch". Clearly not a much exercised
code path, since always false test hasn't worked for two years.
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
redefinition processing was incorrectly using the mismatch file as
an input. Switching to $redefinition_file fixes the processing.
Reported-by: Andrea Adami <andrea.adami@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
createme and kgit-meta had two remaining assumptions about the
meta branch name and directory. With these fixes, all tools will now
respect and use a meta branch name/directory that is set via KMETA
in the environment.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Previously kconfig fragments were required to be in the same
directory as their coupled .scc file. This was to enforce locality
between patches and configuration, but it comes at the cost of
flexibility and the requirement of always having a .scc file plus
a .cfg file to allow re-use.
With this change a "kconf <type> foo.cfg" will search all the
valid include paths that are applied to any .scc feature (which
is typically the in tree kernel-cache, and any external directories
that contain .scc feature descriptions).
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If an include file didn't exist, reporting of the error was
previously delated until the meta series was executed. This
is later than ideal, and can potentially be time consuming.
Moving the check into the file processing time, improves the
timing of the reporting.
There was also latent code for calling 'parent' routines,
but this is no longer used, so it is removed as part of this
improvement.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When an invalid kernel configuration fragment is referenced the
error message did not include the full path to the log, and the
log itself didn't contain the fragment name. This commit fixes
both problems.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It used to be that the branch being built was the most important
bit of information that the tools used to locate the top level
board description. This is no longer the case, and the branch is
largely irrelevant now, so we can delete the routines that locate
.scc files based on branch name.
While working on branches, we add an insurance branch check into
the feature descriptions to ensure that the requested branch
really is the one being built, and where any patches are applied.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
We no longer need kgit-meta to condition the tree when gathering
configuration fragments for config processing, since config
processing is no longer triggered by the branch name (as it
used to be). So we can safely remove the branch manipulation
routines from kgit-meta's config processing.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Integrating the first iteration of a tool that can be used to take
the output of a defconfig configured board, and produce a streamlined
BSP configuration fragment, which only sets hardware and required options.
From the script itself:
kgit-config-cleaner [--force] [--dry-run] [--type <audit type list>]
--audit-dir <dir> -i <input config> -o <output config>
kgit-config-cleaner is run against an input file, and the
results of an initial kernel configuration that has produced a
Kconfig option audit. It produces a modified output file that
can then be fed back into the process to refine a kernel
configuration.
type: comma separated list of audit types to process.
all: the default. process all audit files
errors: process fragment errors
invalid: process invalid options
mismatch: process mismatched options
redefined: process redefined options
non-hardare: process non-hardware options
input config: The input file to be cleaned. This is typically
a defconfig.
output config: The filename of your choice. It is a copy of
input config, with offending lines removed.
audit dir: the directory that contains the output of a yocto
kernel configuration and audit. The input files
in this directory are automatically processed and
used to produce the output
--force: force overwrite the output file
--dry-run: don't modify any files, but dump what would be done
--verbose: run with extra output
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Running kgit-feature on a kernel-cache (or other directory full
of .scc files with feature descriptions), will produce a git
style .rc file that lists all of the features. This rc file can
be processed by external scripts to determine what is available
in a given kernel:
kgit-feature -v -f -o cfg/kern-features.rc cfg/kernel-cache
kgit-feature --dump cfg/kern-features.rc
git show meta:meta/cfg/kern-features.rc > /tmp/kern-rc.rc
git config --list -f /tmp/kern-rc.rc
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Abuse git send-email for the task. Note that the gse in the
sysroot is currently non functional, so we check to make sure
we are _not_ using that one in $PATH 1st.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
During the pre merge processing of fragments and error is thrown
if a fragment doesn't exist. configme wasn't checking the return
code from the sanity script, and hence would continue processing
and produce a .config that would be missing options for any fragment
that followed the one in error.
Catching the error, and propagating it up to the calling build
system exposes this to the user .. who can then fix the problem.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The whole point of an scc file is to be the input that
facilitates the generation of the metadata that describes
a BSP branch (its patches, config etc).
So, once a meta-series exists, going back to the scc file to
retrieve data doesn't make any sense. It is like ld going
back to look for something in a C file.
As we map $SCC --> $META, make both the META_DIR and the
META_BRANCH configurable, since there will be a need for
that going forward. Current defaults are "meta" for both.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
In a case where buildall finds one failure in a larger set, you
may not want to re-run all the passed builds. To that end, being
able to "resume" in an existing buildall space, by retrying the
builds which failed (and skipping those which passed), can be very
handy for finalizing a tree.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Largely due to historical reasons, there used to be a distinction
between the top of a "project" and the location of the kernel tree.
However, we currently were just passing in /dev/null to pad the args
and doing a pointless "if [ -d ... ]" on it.
Since the current instance ends up having `pwd` for both LINUX_SRC
and TOP_BUILD_DIR, just replace all instances of the latter with
the former.
With respect to the /dev/null, replace it with the $META file name.
We want this in kconf_check, since it was storing the audit data
on a per branch basis, and that is no longer unique now that several
BSPs re-use the same branch, resulting in overwritten audit data
when multiple builds were done in a single tree (i.e. buildall).
The same change is made in configme to KTGT; it was set to the
branch, and used in the path of audit data, but it is not unique.
As we aren't using the branch name at all anymore, drop it from
the args entirely. This allows deleting the get_branch function,
and we assign KTGT=$KTYPE/$KMACHINE which should hopefully be
unique per build.
For BSPs that have their own unique branch, the above is consistent
with the current kgit-meta path creation naming behaviour, i.e.:
linux$ls meta/cfg/standard/emenlow/
ls: cannot access meta/cfg/standard/emenlow/: No such file or directory
linux$kgit-meta -v -k meta/cfg/meta/emenlow-standard-meta
[INFO] collecting configs in ./meta/meta-series
[##################################################] (completed in 1 seconds)
linux$ls meta/cfg/standard/emenlow/
config_frag.txt hdw_frags.txt non_hdw_frags.txt
linux$
We see a slightly different behaviour from kgit-meta though,
when the branch is shared. It had similar code that, on _kconf
processing, would use the non-unique branch name for placement
of audit data. It also has been updated to use instead a KTGT
that is set from $KTYPE/$KMACHINE
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Everything configme needs is contained in the meta-series.
Having it go after multiple sources of information can just
lead to confusion, esp. if the meta-series is out of sync
with the scc file specified in meta/top_tgt.
Make configme use the meta series as the authoratative source
of information, and strip out all magic around trying to find
and rate/rank scc files.
This doesn't make use of the added KMACHINE setting that is
parsed out, but it will be used in the future to make the
command line value of $machine redundant/obsolete.
Also make the regex matches on the key defines more strict.
While the scc files can have comments, the meta-series does
not, so in theory we shouldn't need to anchor the regex to BOL
(but do so anyway, just to be 100% safe).
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Make it clear that there is only one input and that it is
expected to be <meta-dir>/meta-series -- the alternative
"go find one please" logic is no longer used or required.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When generating a tree, it is (unfortunately) common that the
base tree doesn't contain the tag that the user thinks it does.
In this case, we should abort the tree generation and report a
clear error.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
This can be combined with RANDOMIZE to get a quick sanity
test on the tree without doing each and every board.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
With 40+ builds (and inevitably more to come), having the fail
logs in the dir alongside of the individual builds is too busy.
Leave them in their respective build directories. This means
dumping the "wc -l" based pass/fail accounting in favour of an
actual counter, which is cleaner anyway.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
For sanity testing, it is nice to be able to build all the
possible kernels with nothing more than the kernel git repo
and a handful of cross compilers.
This script achieves that by determining what all the possible
boards are, what their respective branch is, and then configuring
and compiling each and every one of them.
A sample RC file can be copied to $HOME/.buildallrc and then
customized to suit local variations -- e.g. paths to toolchains,
path to the tree that you want to test, and so on.
The defaults assume you've already got a populated sysroot from
building the main architectures at least once already in order
to provide toolchains, but there is no requirement to use the
yocto generated toolchains from the sysroot.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
We aren't using this, and even if we were, we'd need to add
a "head -n1" since there are scc with multiple KMACHINE vals.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
To prevent invalid args from getting farther than they should.
For example, consider this case where the profile was left out,
i.e. the args should have been "standard chiefriver" and not
just "chiefriver":
linux$~/git/yocto-kernel-tools/tools/configme -o ../chiefriver-standard chiefriver
[INFO] Configuring machine description: chiefriver-standard.scc
[INFO] collecting configs in ./meta/meta-series
[##################################################] (completed in 0 seconds)
[INFO] Pre-processed cfg file -chiefriver-config-3.4.1 created.
[INFO] processing of raw cfg data completed.
It actually runs, because most of the information came from the
scc file via meta/top_tgt, but you can see the config file it
created is wrong (should be "chiefriver-standard-config-3.4.1")
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
scc has gathered some obselete commands over the years. The
ability to indicate that patches should or shouldn't be included
based on certain conditions was supported when scc operated
outside of a git repository. The git based support for this ended
up being complex and error prone. It is no longer used in any
known feature descriptions, so the support can be removed from
scc.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
A construct such as:
if [ "$KARCH" = "x86" ]; then
include feature.scc
fi
wouldn't work, since the variable KARCH was not defined within
the limited environment used to determine the files that make
up a feature series.
Switching the detection of the files list from a subshell that
processes commands, to a simple grep check for 'include' solves
the problem, and simplfies the code.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If we are working on the tools, we want the ones being used to
be treated as a set. So, if we are explicitly calling a configme
from the tool repo, we want merge_config.sh and other tools
to also be used from the repo, and not from whatever version
was captured in-tree on the meta branch.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Without this test, an unset build dir gives the confusing msg:
linux$~/git/yocto-kernel-tools/tools/configme romley
[INFO] Configuring machine description: romley-standard.scc
Dir already exists, remove/move
it if you want to re-run this configuration utility.
config of (romley-standard.scc) failed
linux$
What you cant see above is that it is really trying to say:
Dir "" already exists
i.e. the variable isn't set, and so it tests existence of ./
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Stupid mistakes, like a typo in the path to the kernel-cache repo
can be hard to diagnose, when the error message is only:
phase # 2) cloning source repository
ERROR. No base branch version found, exiting
We can easily augment some of the existing if statements with
an else + warning, so that we now get something like:
WARNING: search dir ~/git/yocto-kernel-cache does not exist
phase # 2) cloning source repository
ERROR. No base branch version found, exiting
and then it is immediately evident what the bad path was.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
The value here is when you use "-j30" vs. "-j 30". The former
will silently fail since this isn't true getopt and you get
a non deterministic usage message. But if you get a message:
Invalid arg "-j30".
as well as the usage message, then chances are you'll clue in andi
be able to suss things out from there.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Out of tree feature descriptions (.scc files) take two forms: normal
features and BSP descriptions.
A normal feature is detected and added to the end of the current machine
being processed. During tree processing, it's configuration and patches
will be applied.
A BSP description on the other hand must be matched based on three
critera (which are in the .scc file via "define <foo>"):
- machine
- kernel type
- architecture
Since features that define machines are only explicitly added, they
are removed from the list of features that should be automatically
added.
The criteria for removing them from the auto-add list is the
definitions found in the .scc file. The existing check was simply
for KMACHINE anywhere in the file. This meant that a conditional
or even a comment containing that phrase would exclude a file.
Properly anchoring the KMACHINE test to "^define.*KMACHINE" fixes the
problem of overly agreesive exclusions.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If an explicit meta branch is not passed, createme should not test for
a directory of the same name to use as the meta directory.
In the scenario where a custom kernel repository is used and no meta
branch name is passed the meta branch name is empty. Testing for a
directory of "" always returns true, and the meta directory name
is set to "" as well.
This results in meta data writes to /cfg, /cfg/kernel-cache, etc, which
result in obvious permission denied errors.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
Bruce Ashfield
Michel Thebeau
Paul Gortmaker
Robert P. J. Day