| Age | Commit message (Collapse) | Author |
|---|
|
Information about a task is not stored in Target objects.
This makes it impossible to correctly operate with the builds
where task is specified.
Storing taks name is an enabler for other fixes in UI and backend
related to restarting builds.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently this module is dereferencing errno.ENOENT but the python module "errno"
is not imported, which causes a crash when fetching from a git repository.
Signed-off-by: Romain Perier <romain.perier@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch fixes the project builds view so it doesn't show
"in progress" builds or builds for other projects.
Note that this also modifies the "all builds" view to use
the same queryset filtering as the project builds. This is
to avoid excluding "in progress" builds more than once, which
is what was happening before.
The patch also has a minor change to ensure that when
displaying the project builds page, only builds for that
project are in the results.
The queryset filtering is now split over several lines so
you can see what's going on.
[YOCTO #8236]
[YOCTO #8187]
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The dependencies of OVERRIDES were not including DEFAULTTUNE in OE-Core.
This is pulled in by a bb.utils.contains() reference which the override
dependency tracking code wasn't accounting for.
This patch ensures we do track contains references too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Right now, OVERRIDES dependency variables set using ??=, e.g. TARGET_ARCH
in OE-Core don't have their dependencies tracked. This is a bug, fix it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sadly its not enough to consider the dependencies of OVERRIDES, we
need to resolve their dependencies and so on recursively. If we don't
do this, some variable can be changed and the resulting data store is
incorrect.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the values that make up OVERRIDES are themselves overridden,
we end up into some horrible circular logic. Unfortunately some
metadata does depend on this functionality.
e.g:
DEFAULTTUNE_virtclass-multilib-xxx = Y
which changes TUNE_ARCH
which changes TARGET_ARCH
which changes OVERRIDES
As a solution, we iterate override expansion until the values don't
change. If we iterate more than 5 times we abort and tell the user to
report the issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We were maintaining state in the form of ud.repochanged to determine whether
we need to write the tarball in the case where the tarball already exists, but
this state is maintained only in memory. If we need to update the git repo,
set ud.repochanged to True, and then are interrupted or killed, the tarball
will then be out of date.
Rather than maintaining this state, simply remove the out of date tarball when
we update the git repo, and it will be re-created with updated content in
build_mirror_data.
[YOCTO #6366]
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A RedirectException is used to redirect the client to the
correct page when their original request is missing the
required parameters (page, orderby etc.). However, the code
is difficult to follow.
Rather than catching RedirectExceptions and rethrowing them with
different view URLs, ensure that the RedirectException has the
correct URL in it when thrown by passing the original page
name to the _build_list_helper() method.
Modified from an original patch by
David Reyna <david.reyna@windriver.com>.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These were referring to the old HTML elements with ids
"date_from_created" and "date_from_updated", but their
ids have changed to "date_from_started_on" and
"date_from_completed_on". This meant that they weren't
functional.
This fixes the references.
Modified from an original patch by David Reyna
<david.reyna@windriver.com>.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The counter for completed builds on the project pages
includes builds in progress.
Instead use the completedbuilds queryset to count the
number of completed builds for display on project pages.
Modify how the completedbuilds queryset is constructed so
it only excludes builds with status "in progress".
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch fixes the project builds view so it doesn't show
"in progress" builds or builds for other projects.
Note that this also modifies the "all builds" view to use
the same queryset filtering as the project builds. This is
to avoid excluding "in progress" builds more than once, which
is what was happening before.
The patch also has a minor change to ensure that when
displaying the project builds page, only builds for that
project are in the results.
The queryset filtering is now split over several lines so
you can see what's going on.
[YOCTO #8236]
[YOCTO #8187]
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of reproducessing the same line over and over and over, we remove the
current line from the mirror list. This permits us to re-evaluate the list
while excluding all matches that have previousily occured.
Without this fix, adding this test results in a failure:
RuntimeError: maximum recursion depth exceeded in cmp
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We should only substitute one time. If we do it without a max count, we can
end up matching over and over.
Before this change:
https://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz
with a mirror of
https://.*/[^/]* http://AAAA/A/A/A/
would end up either recursing indefinitely or result in:
http://AAAA/A/A/A/A/A/A/A/A/A/bitbake-1.0.tar.gz
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Cache the mimetype object and only define the function for
getting a mimetype once.
Also ensure that filemagic is listed as a requirement of
toaster. Doing this also means we can remove the code
which tries multiple different "magic" libraries, as we know
we have the right version available.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The cooker log is copied from its original (bitbake) location
to an artifact directory chosen by the user (chosen when
checksettings.py runs as part of the managed mode;
it's one of the annoying questions you're asked at startup).
The copy happens as part of the runbuilds script run, which
is started in a loop from the toaster startup script
in managed mode.
When a user requests the log for a build via toaster, they
are getting the log which has been copied to the artifact
directory, not the original bitbake log.
This works for the managed case, where the runbuilds command is
running in a loop and copying log files for completed builds to
the artifact directory. However, in analysis mode, there are two
problems:
1. checksettings isn't run, so the artifacts directory isn't
set. toaster is then unable to figure out where the log files
should have been copied to.
2. The log files aren't copied to the artifacts directory
anyway, as runbuilds isn't running in analysis mode.
To fix this, just point the user to the local bitbake log file
in its original location. This avoids the copy step, and means
we can remove a whole question from the toaster startup sequence,
as we no longer need an artifact directory.
The only downside to this is that it's not appropriate for
remote bitbake servers. We will need to revisit this and
possibly reinstate the copy step once we have to reconcile
local and remote builds and make their logs available in
the same way.
[YOCTO #8209]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The BB_CONSOLELOG variable changes by the time we read it in
BuildInfoHelper. This means that the log file location we
are using is incorrect, so the links to the cooker logs don't
work.
Instead, read it at the point when the BuildStarted event occurs
in toasterui. The BB_CONSOLELOG variable has the correct value
here, so pass that to BuildInfoHelper.
[YOCTO #8209]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch cleans up the multiple delete. It:
1) skips build id's that don't exist rather than giving a traceback.
2) let you pass in the ids as a space separated list
3) fixes the usage to match the space separated list format
[YOCTO #7726]
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
No code changes, just refactoring to allow for functionality
changes by moving things to a separate function.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there is a space in a directory name containing a file in file-checksums
(e.g. from a file:// url), you currently get tracebacks from bitbake. This
improves the code to handle colons and spaces in the file-checksums names
since it possible to figure out the correct names.
[YOCTO #8267]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the pipe is closed, we want to ensure that we kill any child processes
by triggering the sigterm handler before we exit. This code does that,
hopefully avoiding the remaining process left behind issues on the autobuilder.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Increased waiting time for toasterUI from 10 to 25 seconds. Bitbake
takes longer time to start, so toaster should wait longer.
[YOCTO #8240]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch:
* Changes the breadcrumb to provide access to either
the project builds or the project configuration, as
appropriate
* Changes the left navigation in the project configuration
to reflect the hierarchical relationship between the
basic configuration and all other configuration pages
* Changes the left navigation in the build history to bring
it in line with the changes in the project configuration
This way the breadcrumb explicitly exposes the hierarchy
of the application, which is its correct behaviour, making it
easier to move around within Toaster.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we SIGKILL cooker (the parent process), ensure the worker notices
and shuts down gracefully. To do this:
* trigger the sigterm handler if the parent exits
* ensure broken pipe writes don't trigger backtraces which interfer with
other exit work
* notice if our command pipe is broken due to EOF and sigterm if so
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
start/end are unused here and we can improve the code conditional blocks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Worryingly, if you SIGKILL the bitbake cooker, an autostarted PR server
will remain behind. It turns out there are a few things we should do:
* The PR service doesn't need to daemonize when started from cooker,
it just complicated the process lifecycle. Add a fork() method
to handle this and use the non-daemon mode for the singleton.
* Reset the sigterm and sigint handlers. Bitbake cooker installs its
own which we inherit meaning PR server was ignoring SIGTERM. Installing
our own handlers which include a sync makes most sense here. Since
we're in the code, make it sync the database on SIGINT.
* Use the new bb.utils.signal_on_parent_exit() call so that we get a
SIGTERM when the parent (usually cooker) exits and we can shutdown
too. Alternatives would be having an open pipe or polling
os.getppid() for changes but this seems more effective.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a new bb.utils.signal_on_parent_exit() function so that a process
can register to recieve a signal when the parent dies. There is no
POSIX standard for this and the implementation is Linux specific.
Alternatives would be having an open pipe or polling os.getppid()
for changes but this seems more effective and less invasive to most
of bitbake's code structure.
We need to be able to determine when parents die to ensure child
processes stop running in a variety of circumstances to avoid
locks being held and ensure clean shutdown.
Roughly based on https://gist.github.com/evansd/2346614
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if you send a SIGTERM to the bitbake UI process, the system basically
hangs if tasks are executing. This is because the server process doesn't
actually try any kind of shutdown before exiting.
This patch trys executing a stateForceShutdown command first, which is
enough to stop any active tasks before the system exits.
I also noticed that terminate can execute multiple times, once at SIGTERM
from the handler and once from the real exit. Double execution leads to
stack traces and potential hangs (writes to dead pipes), so ensure
the code only can run once.
With these fixes, bitbake much more correctly deals with SIGTERM to the
UI process.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The ETA we show for builds in progress is woefully
inaccurate. In the 1.8 release we replaced it with
the % of tasks completed. Somehow, we regressed to
the ETA, so bringing the task % back.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make sure that the project name we show for each build
in the latest builds section of the all builds page:
* Has the same styles as the build, depending on the
build status (in progress, failed or success)
* Links to the project page
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was too much white space above the first
heading in the all builds page. This patch removes
the class that was adding the extra space.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When our clone exists, but is out of date, and the attempt to update it fails,
we don't necessarily want to remove the entire clone, particularly if it's
a large repository.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the main fetch method doesn't support checksums, the user will not be
defining them in the recipe, so we don't want to check them for
premirrors/mirrors either. This ensures that we never error due to missing
checksums on a git mirror tarball.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using an absolute file URI, there's no host, and the path starts with
'/', the dir under ${DL_DIR}/git2/ ends up starting with '.', so is hidden.
Remove any leading '.' to fix this.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Show the user only the portion of the traceback which was from the metadata,
nothing from bitbake's internal calls.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This gives us the needed context of the original ExpansionError, which is
invaluable when we have a chain of function calls in the expansion.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The output of "bitbake, -c listtasks pkg" lists tasks with their real names
(starting with "do_"), but then "bitbake -c do_task" fails, as "do_" always
gets unconditionally prepended to task names. This patch handles this error
by checking whether a task starts with "do_" prior to prepending it with it
when the task runlist is being constructed (and a few other corner cases).
[YOCTO #7818]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Improve the explanation we show to users during set up
when asking them to set the layers checkout directory.
The patch also makes minor changes in text formatting to
improve legibility.
[YOCTO #7740]
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you start and suspend a bitbake execution so the bitbake lock is held,
then try and run "bitbake -w '' X", you will see bitbake return an error exit
code but print no message about what happened at all.
The reason is that the -w option creates a "UI" which swallows the messages. The
code which handles this exit failure mode thinks a UI has printed the messages
and therefore doesn't do so.
This adds in an extra parameter to the UI registration code so that we
can figure out whether its a primary UI or not and base decisions on whether
to display information on that instead. This fixes the error shown above and
some bizarre failures on the Yocto Project Autobuilder.
[YOCTO #8239]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
bitbake utils' movefile is now prone to malform the destination
file with duplicated file name strings. Fixing it to force a file
name append iff the dest argument is a dir not a file name
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Test that the correct landing page redirect is applied, depending
on the state of the default project.
Test that the default project is only shown on the /projects page
if it has at least one build.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The tests expect data to be returned in a particular format,
which doesn't include is_default or the num_builds fields
added to support default project improvements.
Add the extra fields so the tests pass again.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Don't include the default "command line builds" project in
the projects view unless it has builds associated with it.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The landing page currently redirects the user if there are any
projects in the db. Because we now always have at least one
(the default one added by a migration), we always get the redirect.
Change this so that when the user hits the landing page,
we only redirect them to the projects page if there is at least
one user-added project and there are no builds.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Command line builds are associated with a "default project"
(as we currently require a build to have a project). This
acts as a container for builds initiated outside Toaster.
Currently, this project is marked as the default by its ID
being 0. However, this doesn't work with MySQL, as MySQL
won't allow 0 in a foreign key which references an
autoincrement field.
Instead, use an is_default field to track the default Project
for builds initiated outside Toaster.
Add a method to fetch this default project, rather than fetching
a project with a magic ID.
Add this default project in a migration, rather than as a side
effect of a get_or_create() style method.
Also ensure that builds always have a project explicitly assigned
to avoid any magic with a build's project foreign key defaulting to
0 (as it no longer does).
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Two issues prevent this migration from working correctly
with a MySQL back-end:
1. MySQL won't allow a default value to be set for an
AutoField, which is what the migration tries to do
for project_id ("ValueError: The database backend does not accept
0 as a value for AutoField.")
2. When migrations are applied to a MySQL back-end, Django
(via South) attempts a dry run of the migration first: it
applies the forward migration then rolls it back. However, this
migration raises an exception on roll back, which causes the
whole series of migrations to fail.
This commit fixes both issues.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In the new build button form, make sure we show a
warning when you select a project with an incomplete
configuration that cannot be built.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make sure that the 'build' button in the 'new build'
form is disabled whenever the 'recipe(s)' text
field is empty.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When moving a file via the os.rename function, it was missing the
destination file name which caused an OSError
[YOCTO#8180]
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
