| Age | Commit message (Collapse) | Author |
|---|
|
We only need to migrate a directory (and its subdirs) once. To improve
performance, we can check to see if a patch is present in the destination
directory structure, and if present, there's no need to do a migration.
Similarly, we can avoid adding duplicate patches, but still have them
processed, by checking for a patch being in a .scc file and only generating
if it isn't already described by a feature.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In a system where the kernel source and object directories are
split we want the generated .config to be in the build directory
and not the current directory. The easiest way to support this is
to respect the O= environment variable.
Neither KBUILD_OUTPUT or O were propagating automatically to the
kernel build, so always explicitly setting O= and passing it to
the kernel build ensures that it is respected. Without this .config
always ends up in the current working directory.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Some environments export variables to subshells and in those
environments setting 'MAKE=true' will interfere with the kernel
build system's use of that same variable. To avoid this, we can
simply rename MAKE to MAKE_FLAG.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When debugging options that are output as not being in the final
.config file it is often useful to look at the fully concatenated
configuration file. To support this we add a debug flag that will
not cleanup the .tmp.config* file that is created. The risk of
having uncleaned .tmp.config* file is an acceptable tradeoff for
this debug assist.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
syncing the kern-tools variant of merge_config.sh to the upstream
kernel version, commit: 409f117e2d.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Updateme is responsible for updating an existing meta-series with new patches,
configs and tree manipulations. To do this, it first checks for an existing
board description and generates one if required. It then searches for features
and fragments to be applied for the tree.
There were two problems:
- A top level board description is detected via the presence of "define"
directives that indicate the board name, the arch and kernel type. The
test for define would match on patches or fragments with 'define' in their
name, and would incorrectly use that file as the top level board description.
This is fixed by ensuring that only defines at the start of a line, or preceded
by whitepace match.
- When searching for features that were indicated as 'addon' or 'optional', the
search would find, and apply, any feature with the passed name as substring
versus an exact match.
This is fixed by ensuring that the matched feature name is /<feature name>
versus <feature name>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When a custom BSP is used, a top level BSP is generated by the tools and fed
to the build system just as a user defined BSP would be located and
passed. The location of the generated file is placed in the top_tgt file,
which is used by subsequent stages. A relative path was being placed into
top_tgt, which binds the build to a particular directory structure and
working directory.
The location of parts of the build have changed, and this relative path is
no longer accurate. Changing it to an absolute path solve the build issues
related to custom BSPs.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The regex that detected flags passed to includes was not specific
enough, and unfortunately named feature files would matchc. and have
their configuration inhibited. This resulted in features like
standard-nocfg.scc inhibiting all configuration items, even base
configs.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
A regression was reported that unless a patch file was local to the .scc
file, it could no longer be found. This is due to the searching for
patch/config files not taking into account a locally passed set of include
paths versus the global ones set via -I <path>
This error could have been masked due to an exit code not propagating
to the final exit code of 'spp', so we fix that as well in this change.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Variables defined in .scc files have two purposes:
- Documentation in the meta-series
- Variables that can be tested in sub sections and other features
The split of functionality into two phases, dropped the second
purpose. As a result, arch tests were failing and configs were dropped.
This restores the existing functionality.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To support inhibiting of configuration values via include directives,
it must happen in a depth first based processing mode. The existing
monolithic processing of files, which then generate a script, that
generates a meta-series work in a model where all files are found,
individually processed and then re-assembled to generate the file output.
To allow enhanced processing of includes, we break the processing into
two parts, while keeping the output identical to the existing outputs
of the scripts. The first phase preprocesses the target files, in a
depth first manner, expanding included files, and converting kernel
configuration and patch directives to absolute paths (while performing
checks on their existence). The second phase, consumes the preprocessed
file, and generates a meta-series for future processing.
The logic and output of the existing scc processing is maintained, but
is now faster, robust and can handle more complex conditioning of the
input files.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When patches are prep'd for tree application they are symlinked
from their original location into a branch named directory. The symlink
assumed the patch would be accesible via the "meta" directory, but meta
can be different per META branch.
Changing the symlink code to use the proper variable, versus the hardcoded
name allows patching to work on a non-meta named directory.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
|
|
The list of tools captured in the meta directory of a new repository
was missing some recent addition. With this change, all tools used
to construct, configure and build a kernel are copied into the
meta/scripts directory during repository creation.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
One of the functions of the kernel configuration audit is to notify
the user if Kconfig* files have been removed from the kernel, and
also to notify of new Kconfig files.
New Kconfig files should be classified as hardware or non-hardware to
allow BSP audits to notify if boards are setting values that they
shouldn't, hence why notifying about new "buckets" is important.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It is common to need the features (patches, git operations) of a
branch, but not want the kernel configuration fragments of a given
branch. To allow this, we provide a new include flag "nocfg".
When this flag is used, all of the configuration fragments included
by the targetted feature will not be applied to the current build,
with one exception, a base/critical fragment can force it's config
values, since without them, the system would not be functional.
Example:
include ktypes/standard/standard.scc nocfg
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To allow for parameter validation and sanity checking, wrap "git merge"
as a dedicated "merge" command instead of using the raw git fallback.
This also makes it consistent with existing top level commands such
as 'tag', 'branch', 'patch', etc.
There are no changes to arguments, and existing 'git merge' commands
continue to work with this change.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If external feature directories are passed to the build, we need to
ensure that internal/in tree ones are prioritized in case of name
space collisions. The fix is simple, we put internal directories to
the front of the include path, instead of on the end.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In cases where a configuration includes repeated include directories
the existing source file searching would iterate and return the same file
multiple times. We only want to return the first matching file found, otherwise
long loops and wasted compiles will be queued.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If a configuration fragment was missing, the previous error output
was not clear about the error:
| [INFO] doing kernel configme
| [INFO] Configuring target/machine combo: "standard/atom-pc"
| [INFO] collecting configs in ./meta/meta-series
| ERROR: could not sanitize configuration fragments
| errors are logged in ... linux/meta/cfg/standard/atom-pc/config.log
but we know the name of the missing fragment and can improve the error
message to be this:
| [ERROR] kernel configuration fragment fragment 'virto.cfg' cannot be found
| ERROR. A meta series could not be created for branch yocto/standard/common-pc/atom-pc
| ERROR. Could not locate meta series for atom-pc
| ERROR. Could not apply patches for atom-pc.
| Patch failures can be resolved in the devshell (bitbake -c devshell linux-yocto)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When searching for the best matching board description the scripts
know how many defines are passed in. The .scc files are then searched
for the one which matches that definition.
The input defines were previously used to calculate both a minimum
score, which was then compared against the maximum score from a
given .scc file. This meant that the best match was not found if
only a single define matched. Having the ability to find the
description that has the highest score, but doesn't match all the
defines is useful when working with repositories with custom
meta data, since things like kernel type are not always defined
or relevant.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It is common that a feature fragment is included multiple times,
and these double inclusions can cause problems if a fragment includes
both patches and configuration.
This was previously dealt with by splitting the feature into an
'enable' configuration fragment and a 'feature' fragment that applies
the patches. This is effective, but requires that knowledge of the
entire include structure be known when creating a new feature.
We can detect when a patch has already been applied, or when a
feature fragment is included multiple times and do two things:
- emit a clear message about the multiple includes
- skip the double application of the patches via tracking a global
series file
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To allow a BSP to verify that critical options have made it to the
final .config (and to throw an error if they are not present), a
new "verify" kernel config fragment type is introduced.
The format of this fragment is identical to any other fragment,
except that its contents will not be used to directly configure
the kernel, but will be checked against the final .config any
missing or changed options will result in a message and error exit
code.
This allows the BSP to enforce critical options, but not have to
override policy or declare non-hardware options as hardware.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Checks for invalid options previously only applied to 'hardware'
options. But with a pending move to required/optional config
buckets, having this check apply to all inputs makes sense.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Minor cleanups to spelling and ordering of some actions. The
functional change is the ability to pass V=1 in the environment
and have the configuration report output additional information
about redefined config values.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When searching for all available Kconfig files, kconf_check was using
$meta_dir instead of $META_DIR. This resulted in a truncated path and
the following warning:
find: warning: -path $oe-path/linux/ will not match anything because it ends with /.
Using the proper variable removes the warning and make sure that we
do actually search all relevant directories.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kconf_check is called from many different types of BSPs and
configuration architectures. As such, not all of the categories of
configuration groups will always be present (i.e. no hardware frags,
no required frags, etc).
To ensure that the varous cat, grep and sort lines to not try and
access non-existent files, we add extra checks, or touches to
files to ensure a consistent set of files before the various checking
phases run.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kernel branches are constructed during patching of the tree by
constructing a '/' based hierarchy of names as each branch
directive is encountered.
But if a "branch $name $branchpoint" is used, the entire branch
name is supplied so no additions to the hierarchy should
happen. As such, that type of branch command should not be part
of branch name calculation and preparation.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To make the generated feature file more useful to external scripts
that wish to offer a list of optional features supported by a kernel,
extra divisions are required in the kernel features rc file.
With this change, there are three categories:
- feature: features that are built into the kernel (not selectable)
- feature-optional: optional (and selectable) features in the kernel
- config: configuration features supported by the kernel
They can be queried like this:
% git config -f /tmp/kern-features.rc --get-all kernel-options.feature-optional
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When a series of patches fail to apply the pending patches are dumped
to the user. But if the GUILT_BASE directory is not set, you simply
get something that looks like an internal error:
| To force apply this patch, use 'guilt push -f'
| [ERROR] unable to complete push
| pending patches are:
| Patches directory doesn't exist, try guilt-init
If we export GUILT_BASE, the output is much better:
| [ERROR] unable to complete push
| pending patches are:
links/files/0002-makefile-patch.patch
| ERROR. Could not update git tree
| ERROR. Could not apply updates for qemux86.
| Patch failures can be resolved in the devshell:
| % bitbake -c devshell linux-yocto
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When forcing a branch switch to the desired branch we should be
using the proper 'branch' command. Since without this call, the
proper variables will not be set, and patches can't be applied
to the tree.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In the case of a custom repository, we may not enounter ANY explicit
branch points. This means that the first patch push will fail, since
the series file and other infrastructure has not been set up. To
ensure that we can always push patches, the starting branch is now
prepped for modifications.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When an explicit branch point is specified:
branch $new_branch $branch_point
it means that the name of the new branch is explicit (and
known), and the branch name shouldn't inherit the names of
the branches that have proceeded it.
So if this is detected, branch name generation is skipped.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The warning:
warning: could not find (or was already included):
Sometimes is not clear because the include path is omitted in scc. So
print the current feature directory as well.
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
|
|
Add the ability to specify a "keep" config file. This gives the
ability to force an option to be kept around, even if it violates
the cleaning rules.
From the help text:
An input config file with a list of configurations that should
be maintained regardless of other cleaning activities. These
are typically config options that should be declared as
'hardware' for the given board, and the keep entry would no
longer be necessary.
Note: This file is normally called 'hardware.cfg', so it can be
reused by the build process in future iterations.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
This is a partial revert of commit 5b6498c4bc02a
buildall: remove all instances of it using/reading scc files
The above was done, since the final branch used by an scc
file is not obvious (imagine scc A that includes scc B which
in turn does the actual branch selection). Since the output
of scc (the meta-series) clearly shows the final branch, the
switch over seemed obvious.
In the context of building all BSPs from a freshly generated
tree, we know the meta series files match what is on all the
branches. However if someone wants to use buildall on a
tree that is in maintenance mode, there may have been new
scc added (i.e. no meta), or configuration fragments relocated
(meta with stale paths) resulting in odd failures that would
not happen if the branch was built via the complete build
system.
This puts things back to being driven from the scc leaf nodes.
The stale meta problem is handled by regenerating them. There
is an optional "use what is there" knob that can be used by
people generating a fresh tree, since they know that there is
no risk of staleness in that one particular use case (and since
regeneration of them can be timewise expensive).
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
Not all features will create branches with the same origin, to support
these features, we need to do several things:
- condition the branch name to be the first argument in "branch <foo> <bar>"
since <foo> is the name and <bar> is the branch point
- during tree generation the branch should be reset to a known starting
point between features.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
script.
Since the only current call to "createme" from do_patch() doesn't pass
a defconfig argument, remove the apparently superfluous references to
defconfig from createme, but leave the check_defconfig() function in
just in case it ever becomes useful.
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Fixing a long standing typo in a test for the verify_branch, which
was "verify_branch" vs "$verify_branch". Clearly not a much exercised
code path, since always false test hasn't worked for two years.
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
redefinition processing was incorrectly using the mismatch file as
an input. Switching to $redefinition_file fixes the processing.
Reported-by: Andrea Adami <andrea.adami@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
createme and kgit-meta had two remaining assumptions about the
meta branch name and directory. With these fixes, all tools will now
respect and use a meta branch name/directory that is set via KMETA
in the environment.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Previously kconfig fragments were required to be in the same
directory as their coupled .scc file. This was to enforce locality
between patches and configuration, but it comes at the cost of
flexibility and the requirement of always having a .scc file plus
a .cfg file to allow re-use.
With this change a "kconf <type> foo.cfg" will search all the
valid include paths that are applied to any .scc feature (which
is typically the in tree kernel-cache, and any external directories
that contain .scc feature descriptions).
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If an include file didn't exist, reporting of the error was
previously delated until the meta series was executed. This
is later than ideal, and can potentially be time consuming.
Moving the check into the file processing time, improves the
timing of the reporting.
There was also latent code for calling 'parent' routines,
but this is no longer used, so it is removed as part of this
improvement.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When an invalid kernel configuration fragment is referenced the
error message did not include the full path to the log, and the
log itself didn't contain the fragment name. This commit fixes
both problems.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It used to be that the branch being built was the most important
bit of information that the tools used to locate the top level
board description. This is no longer the case, and the branch is
largely irrelevant now, so we can delete the routines that locate
.scc files based on branch name.
While working on branches, we add an insurance branch check into
the feature descriptions to ensure that the requested branch
really is the one being built, and where any patches are applied.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
We no longer need kgit-meta to condition the tree when gathering
configuration fragments for config processing, since config
processing is no longer triggered by the branch name (as it
used to be). So we can safely remove the branch manipulation
routines from kgit-meta's config processing.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Integrating the first iteration of a tool that can be used to take
the output of a defconfig configured board, and produce a streamlined
BSP configuration fragment, which only sets hardware and required options.
From the script itself:
kgit-config-cleaner [--force] [--dry-run] [--type <audit type list>]
--audit-dir <dir> -i <input config> -o <output config>
kgit-config-cleaner is run against an input file, and the
results of an initial kernel configuration that has produced a
Kconfig option audit. It produces a modified output file that
can then be fed back into the process to refine a kernel
configuration.
type: comma separated list of audit types to process.
all: the default. process all audit files
errors: process fragment errors
invalid: process invalid options
mismatch: process mismatched options
redefined: process redefined options
non-hardare: process non-hardware options
input config: The input file to be cleaned. This is typically
a defconfig.
output config: The filename of your choice. It is a copy of
input config, with offending lines removed.
audit dir: the directory that contains the output of a yocto
kernel configuration and audit. The input files
in this directory are automatically processed and
used to produce the output
--force: force overwrite the output file
--dry-run: don't modify any files, but dump what would be done
--verbose: run with extra output
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
Bruce Ashfield
Bruce Ashfield
Robert P. J. Day
Michel Thebeau
Paul Gortmaker