| Age | Commit message (Collapse) | Author |
|---|
|
To support inhibiting of configuration values via include directives,
it must happen in a depth first based processing mode. The existing
monolithic processing of files, which then generate a script, that
generates a meta-series work in a model where all files are found,
individually processed and then re-assembled to generate the file output.
To allow enhanced processing of includes, we break the processing into
two parts, while keeping the output identical to the existing outputs
of the scripts. The first phase preprocesses the target files, in a
depth first manner, expanding included files, and converting kernel
configuration and patch directives to absolute paths (while performing
checks on their existence). The second phase, consumes the preprocessed
file, and generates a meta-series for future processing.
The logic and output of the existing scc processing is maintained, but
is now faster, robust and can handle more complex conditioning of the
input files.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When patches are prep'd for tree application they are symlinked
from their original location into a branch named directory. The symlink
assumed the patch would be accesible via the "meta" directory, but meta
can be different per META branch.
Changing the symlink code to use the proper variable, versus the hardcoded
name allows patching to work on a non-meta named directory.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
|
|
The list of tools captured in the meta directory of a new repository
was missing some recent addition. With this change, all tools used
to construct, configure and build a kernel are copied into the
meta/scripts directory during repository creation.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
One of the functions of the kernel configuration audit is to notify
the user if Kconfig* files have been removed from the kernel, and
also to notify of new Kconfig files.
New Kconfig files should be classified as hardware or non-hardware to
allow BSP audits to notify if boards are setting values that they
shouldn't, hence why notifying about new "buckets" is important.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It is common to need the features (patches, git operations) of a
branch, but not want the kernel configuration fragments of a given
branch. To allow this, we provide a new include flag "nocfg".
When this flag is used, all of the configuration fragments included
by the targetted feature will not be applied to the current build,
with one exception, a base/critical fragment can force it's config
values, since without them, the system would not be functional.
Example:
include ktypes/standard/standard.scc nocfg
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To allow for parameter validation and sanity checking, wrap "git merge"
as a dedicated "merge" command instead of using the raw git fallback.
This also makes it consistent with existing top level commands such
as 'tag', 'branch', 'patch', etc.
There are no changes to arguments, and existing 'git merge' commands
continue to work with this change.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If external feature directories are passed to the build, we need to
ensure that internal/in tree ones are prioritized in case of name
space collisions. The fix is simple, we put internal directories to
the front of the include path, instead of on the end.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In cases where a configuration includes repeated include directories
the existing source file searching would iterate and return the same file
multiple times. We only want to return the first matching file found, otherwise
long loops and wasted compiles will be queued.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If a configuration fragment was missing, the previous error output
was not clear about the error:
| [INFO] doing kernel configme
| [INFO] Configuring target/machine combo: "standard/atom-pc"
| [INFO] collecting configs in ./meta/meta-series
| ERROR: could not sanitize configuration fragments
| errors are logged in ... linux/meta/cfg/standard/atom-pc/config.log
but we know the name of the missing fragment and can improve the error
message to be this:
| [ERROR] kernel configuration fragment fragment 'virto.cfg' cannot be found
| ERROR. A meta series could not be created for branch yocto/standard/common-pc/atom-pc
| ERROR. Could not locate meta series for atom-pc
| ERROR. Could not apply patches for atom-pc.
| Patch failures can be resolved in the devshell (bitbake -c devshell linux-yocto)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When searching for the best matching board description the scripts
know how many defines are passed in. The .scc files are then searched
for the one which matches that definition.
The input defines were previously used to calculate both a minimum
score, which was then compared against the maximum score from a
given .scc file. This meant that the best match was not found if
only a single define matched. Having the ability to find the
description that has the highest score, but doesn't match all the
defines is useful when working with repositories with custom
meta data, since things like kernel type are not always defined
or relevant.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It is common that a feature fragment is included multiple times,
and these double inclusions can cause problems if a fragment includes
both patches and configuration.
This was previously dealt with by splitting the feature into an
'enable' configuration fragment and a 'feature' fragment that applies
the patches. This is effective, but requires that knowledge of the
entire include structure be known when creating a new feature.
We can detect when a patch has already been applied, or when a
feature fragment is included multiple times and do two things:
- emit a clear message about the multiple includes
- skip the double application of the patches via tracking a global
series file
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To allow a BSP to verify that critical options have made it to the
final .config (and to throw an error if they are not present), a
new "verify" kernel config fragment type is introduced.
The format of this fragment is identical to any other fragment,
except that its contents will not be used to directly configure
the kernel, but will be checked against the final .config any
missing or changed options will result in a message and error exit
code.
This allows the BSP to enforce critical options, but not have to
override policy or declare non-hardware options as hardware.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Checks for invalid options previously only applied to 'hardware'
options. But with a pending move to required/optional config
buckets, having this check apply to all inputs makes sense.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Minor cleanups to spelling and ordering of some actions. The
functional change is the ability to pass V=1 in the environment
and have the configuration report output additional information
about redefined config values.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When searching for all available Kconfig files, kconf_check was using
$meta_dir instead of $META_DIR. This resulted in a truncated path and
the following warning:
find: warning: -path $oe-path/linux/ will not match anything because it ends with /.
Using the proper variable removes the warning and make sure that we
do actually search all relevant directories.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kconf_check is called from many different types of BSPs and
configuration architectures. As such, not all of the categories of
configuration groups will always be present (i.e. no hardware frags,
no required frags, etc).
To ensure that the varous cat, grep and sort lines to not try and
access non-existent files, we add extra checks, or touches to
files to ensure a consistent set of files before the various checking
phases run.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kernel branches are constructed during patching of the tree by
constructing a '/' based hierarchy of names as each branch
directive is encountered.
But if a "branch $name $branchpoint" is used, the entire branch
name is supplied so no additions to the hierarchy should
happen. As such, that type of branch command should not be part
of branch name calculation and preparation.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To make the generated feature file more useful to external scripts
that wish to offer a list of optional features supported by a kernel,
extra divisions are required in the kernel features rc file.
With this change, there are three categories:
- feature: features that are built into the kernel (not selectable)
- feature-optional: optional (and selectable) features in the kernel
- config: configuration features supported by the kernel
They can be queried like this:
% git config -f /tmp/kern-features.rc --get-all kernel-options.feature-optional
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When a series of patches fail to apply the pending patches are dumped
to the user. But if the GUILT_BASE directory is not set, you simply
get something that looks like an internal error:
| To force apply this patch, use 'guilt push -f'
| [ERROR] unable to complete push
| pending patches are:
| Patches directory doesn't exist, try guilt-init
If we export GUILT_BASE, the output is much better:
| [ERROR] unable to complete push
| pending patches are:
links/files/0002-makefile-patch.patch
| ERROR. Could not update git tree
| ERROR. Could not apply updates for qemux86.
| Patch failures can be resolved in the devshell:
| % bitbake -c devshell linux-yocto
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When forcing a branch switch to the desired branch we should be
using the proper 'branch' command. Since without this call, the
proper variables will not be set, and patches can't be applied
to the tree.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In the case of a custom repository, we may not enounter ANY explicit
branch points. This means that the first patch push will fail, since
the series file and other infrastructure has not been set up. To
ensure that we can always push patches, the starting branch is now
prepped for modifications.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When an explicit branch point is specified:
branch $new_branch $branch_point
it means that the name of the new branch is explicit (and
known), and the branch name shouldn't inherit the names of
the branches that have proceeded it.
So if this is detected, branch name generation is skipped.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The warning:
warning: could not find (or was already included):
Sometimes is not clear because the include path is omitted in scc. So
print the current feature directory as well.
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
|
|
Add the ability to specify a "keep" config file. This gives the
ability to force an option to be kept around, even if it violates
the cleaning rules.
From the help text:
An input config file with a list of configurations that should
be maintained regardless of other cleaning activities. These
are typically config options that should be declared as
'hardware' for the given board, and the keep entry would no
longer be necessary.
Note: This file is normally called 'hardware.cfg', so it can be
reused by the build process in future iterations.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
This is a partial revert of commit 5b6498c4bc02a
buildall: remove all instances of it using/reading scc files
The above was done, since the final branch used by an scc
file is not obvious (imagine scc A that includes scc B which
in turn does the actual branch selection). Since the output
of scc (the meta-series) clearly shows the final branch, the
switch over seemed obvious.
In the context of building all BSPs from a freshly generated
tree, we know the meta series files match what is on all the
branches. However if someone wants to use buildall on a
tree that is in maintenance mode, there may have been new
scc added (i.e. no meta), or configuration fragments relocated
(meta with stale paths) resulting in odd failures that would
not happen if the branch was built via the complete build
system.
This puts things back to being driven from the scc leaf nodes.
The stale meta problem is handled by regenerating them. There
is an optional "use what is there" knob that can be used by
people generating a fresh tree, since they know that there is
no risk of staleness in that one particular use case (and since
regeneration of them can be timewise expensive).
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Not all features will create branches with the same origin, to support
these features, we need to do several things:
- condition the branch name to be the first argument in "branch <foo> <bar>"
since <foo> is the name and <bar> is the branch point
- during tree generation the branch should be reset to a known starting
point between features.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
script.
Since the only current call to "createme" from do_patch() doesn't pass
a defconfig argument, remove the apparently superfluous references to
defconfig from createme, but leave the check_defconfig() function in
just in case it ever becomes useful.
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Fixing a long standing typo in a test for the verify_branch, which
was "verify_branch" vs "$verify_branch". Clearly not a much exercised
code path, since always false test hasn't worked for two years.
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
redefinition processing was incorrectly using the mismatch file as
an input. Switching to $redefinition_file fixes the processing.
Reported-by: Andrea Adami <andrea.adami@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
createme and kgit-meta had two remaining assumptions about the
meta branch name and directory. With these fixes, all tools will now
respect and use a meta branch name/directory that is set via KMETA
in the environment.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Previously kconfig fragments were required to be in the same
directory as their coupled .scc file. This was to enforce locality
between patches and configuration, but it comes at the cost of
flexibility and the requirement of always having a .scc file plus
a .cfg file to allow re-use.
With this change a "kconf <type> foo.cfg" will search all the
valid include paths that are applied to any .scc feature (which
is typically the in tree kernel-cache, and any external directories
that contain .scc feature descriptions).
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If an include file didn't exist, reporting of the error was
previously delated until the meta series was executed. This
is later than ideal, and can potentially be time consuming.
Moving the check into the file processing time, improves the
timing of the reporting.
There was also latent code for calling 'parent' routines,
but this is no longer used, so it is removed as part of this
improvement.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When an invalid kernel configuration fragment is referenced the
error message did not include the full path to the log, and the
log itself didn't contain the fragment name. This commit fixes
both problems.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It used to be that the branch being built was the most important
bit of information that the tools used to locate the top level
board description. This is no longer the case, and the branch is
largely irrelevant now, so we can delete the routines that locate
.scc files based on branch name.
While working on branches, we add an insurance branch check into
the feature descriptions to ensure that the requested branch
really is the one being built, and where any patches are applied.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
We no longer need kgit-meta to condition the tree when gathering
configuration fragments for config processing, since config
processing is no longer triggered by the branch name (as it
used to be). So we can safely remove the branch manipulation
routines from kgit-meta's config processing.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Integrating the first iteration of a tool that can be used to take
the output of a defconfig configured board, and produce a streamlined
BSP configuration fragment, which only sets hardware and required options.
From the script itself:
kgit-config-cleaner [--force] [--dry-run] [--type <audit type list>]
--audit-dir <dir> -i <input config> -o <output config>
kgit-config-cleaner is run against an input file, and the
results of an initial kernel configuration that has produced a
Kconfig option audit. It produces a modified output file that
can then be fed back into the process to refine a kernel
configuration.
type: comma separated list of audit types to process.
all: the default. process all audit files
errors: process fragment errors
invalid: process invalid options
mismatch: process mismatched options
redefined: process redefined options
non-hardare: process non-hardware options
input config: The input file to be cleaned. This is typically
a defconfig.
output config: The filename of your choice. It is a copy of
input config, with offending lines removed.
audit dir: the directory that contains the output of a yocto
kernel configuration and audit. The input files
in this directory are automatically processed and
used to produce the output
--force: force overwrite the output file
--dry-run: don't modify any files, but dump what would be done
--verbose: run with extra output
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Running kgit-feature on a kernel-cache (or other directory full
of .scc files with feature descriptions), will produce a git
style .rc file that lists all of the features. This rc file can
be processed by external scripts to determine what is available
in a given kernel:
kgit-feature -v -f -o cfg/kern-features.rc cfg/kernel-cache
kgit-feature --dump cfg/kern-features.rc
git show meta:meta/cfg/kern-features.rc > /tmp/kern-rc.rc
git config --list -f /tmp/kern-rc.rc
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Abuse git send-email for the task. Note that the gse in the
sysroot is currently non functional, so we check to make sure
we are _not_ using that one in $PATH 1st.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
During the pre merge processing of fragments and error is thrown
if a fragment doesn't exist. configme wasn't checking the return
code from the sanity script, and hence would continue processing
and produce a .config that would be missing options for any fragment
that followed the one in error.
Catching the error, and propagating it up to the calling build
system exposes this to the user .. who can then fix the problem.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The whole point of an scc file is to be the input that
facilitates the generation of the metadata that describes
a BSP branch (its patches, config etc).
So, once a meta-series exists, going back to the scc file to
retrieve data doesn't make any sense. It is like ld going
back to look for something in a C file.
As we map $SCC --> $META, make both the META_DIR and the
META_BRANCH configurable, since there will be a need for
that going forward. Current defaults are "meta" for both.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
In a case where buildall finds one failure in a larger set, you
may not want to re-run all the passed builds. To that end, being
able to "resume" in an existing buildall space, by retrying the
builds which failed (and skipping those which passed), can be very
handy for finalizing a tree.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Largely due to historical reasons, there used to be a distinction
between the top of a "project" and the location of the kernel tree.
However, we currently were just passing in /dev/null to pad the args
and doing a pointless "if [ -d ... ]" on it.
Since the current instance ends up having `pwd` for both LINUX_SRC
and TOP_BUILD_DIR, just replace all instances of the latter with
the former.
With respect to the /dev/null, replace it with the $META file name.
We want this in kconf_check, since it was storing the audit data
on a per branch basis, and that is no longer unique now that several
BSPs re-use the same branch, resulting in overwritten audit data
when multiple builds were done in a single tree (i.e. buildall).
The same change is made in configme to KTGT; it was set to the
branch, and used in the path of audit data, but it is not unique.
As we aren't using the branch name at all anymore, drop it from
the args entirely. This allows deleting the get_branch function,
and we assign KTGT=$KTYPE/$KMACHINE which should hopefully be
unique per build.
For BSPs that have their own unique branch, the above is consistent
with the current kgit-meta path creation naming behaviour, i.e.:
linux$ls meta/cfg/standard/emenlow/
ls: cannot access meta/cfg/standard/emenlow/: No such file or directory
linux$kgit-meta -v -k meta/cfg/meta/emenlow-standard-meta
[INFO] collecting configs in ./meta/meta-series
[##################################################] (completed in 1 seconds)
linux$ls meta/cfg/standard/emenlow/
config_frag.txt hdw_frags.txt non_hdw_frags.txt
linux$
We see a slightly different behaviour from kgit-meta though,
when the branch is shared. It had similar code that, on _kconf
processing, would use the non-unique branch name for placement
of audit data. It also has been updated to use instead a KTGT
that is set from $KTYPE/$KMACHINE
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Everything configme needs is contained in the meta-series.
Having it go after multiple sources of information can just
lead to confusion, esp. if the meta-series is out of sync
with the scc file specified in meta/top_tgt.
Make configme use the meta series as the authoratative source
of information, and strip out all magic around trying to find
and rate/rank scc files.
This doesn't make use of the added KMACHINE setting that is
parsed out, but it will be used in the future to make the
command line value of $machine redundant/obsolete.
Also make the regex matches on the key defines more strict.
While the scc files can have comments, the meta-series does
not, so in theory we shouldn't need to anchor the regex to BOL
(but do so anyway, just to be 100% safe).
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Make it clear that there is only one input and that it is
expected to be <meta-dir>/meta-series -- the alternative
"go find one please" logic is no longer used or required.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When generating a tree, it is (unfortunately) common that the
base tree doesn't contain the tag that the user thinks it does.
In this case, we should abort the tree generation and report a
clear error.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
This can be combined with RANDOMIZE to get a quick sanity
test on the tree without doing each and every board.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
Bruce Ashfield
Bruce Ashfield
Robert P. J. Day
Michel Thebeau
Paul Gortmaker