summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* Create a test case for the poll loop which uses the loop to read data from aZac Medico2008-07-124-41/+272
| | | | | | | | | | | | | | | | | | | | | | | | | | pipe and assert that the data written to the pipe is identical to the data read from the pipe. In order to implement this test, several useful classes have been added: * PipeReader Reads output from one or more files and saves it in memory, for retrieval via the getvalue() method. This is driven by the scheduler's poll() loop, so it runs entirely within the current process. * QueueScheduler Add instances of SequentialTaskQueue and then call run(). The run() method returns when no tasks remain. * TaskScheduler A simple way to handle scheduling of AsynchrousTask instances. Simply add tasks and call run(). The run() method returns when no tasks remain. svn path=/main/trunk/; revision=11022
* * Add a digraph.get() method, analogous to dict.get(), used to simplifyZac Medico2008-07-122-6/+7
| | | | | | | | Scheduler._pkg(). * Define digraph.__contains__ for containment tests via the "in" operator. svn path=/main/trunk/; revision=11021
* For subclasses of Task, explicitly assign __hash__ = Task.__hash__ forZac Medico2008-07-121-0/+5
| | | | | | compatibility with python-2.6_beta. Thanks to Arfrever for reporting. svn path=/main/trunk/; revision=11020
* Fix SpawnProcess.start() to return early and notify exit listeners if theZac Medico2008-07-111-1/+10
| | | | | | spawn call fails. svn path=/main/trunk/; revision=11019
* Fix a few typos found with pyflakes. Thanks to Cardoe for reporting.Zac Medico2008-07-111-3/+5
| | | | svn path=/main/trunk/; revision=11018
* Copy INHERITED/_eclasses_ handling from portdbapi._metadata_callback() toZac Medico2008-07-111-0/+6
| | | | | | | portdbapi.aux_get() since it's still needed there or else aux_get won't properly return INHERITED. svn path=/main/trunk/; revision=11016
* Call settings.reset() in EbuildMetadataPhase.start() since that's whatZac Medico2008-07-111-0/+1
| | | | | | | | portdbapi does too before generating metadata, and without it, a previous CATEGORY can leak into the ebuild env and corrupt it (breaks lots of packages that reference $CATEGORY). svn path=/main/trunk/; revision=11015
* Document the new --jobs and --load-average options for parallelization.Zac Medico2008-07-113-5/+37
| | | | svn path=/main/trunk/; revision=11014
* Refactor and simplify the main task scheduling and poll loops:Zac Medico2008-07-111-94/+74
| | | | | | | | | | | | | | * Make output handlers unregister themselves and call wait() to notify exit listeners immediately. This makes the exit listeners more useful for scheduling tasks. This makes the poll loop nice an clean because it just calls the handlers and then the handlers can do the scheduling when necessary. * Make SequentialTaskQueue.add() and addFront() trigger scheduling internally, so that it's more of a chain reaction than something that has to be done explicitly. svn path=/main/trunk/; revision=11013
* Add support for parallel --regen, using the existing --jobs and --load-averageZac Medico2008-07-103-202/+434
| | | | | | | | | | | | | | | | options: * Split out a PollLoop base class from Scheduler and derive a MetadataRegen class to handle parallel scheduling for --regen. * Add pordbapi._metadata_process() and _metadata_callback() methods to implement asynchronous metadata generation. One method returns an EbuildMetadataPhase instance to encapsulate the async task. The other method is called to save the metadata when the process completes successfully. These methods share code with aux_get to avoid duplicate code. svn path=/main/trunk/; revision=11012
* When a package replaces another one, add the corresponding uninstall taskZac Medico2008-07-101-1/+19
| | | | | | | for the replaced package to Scheduler._completed_tasks so that it's properly accounted for in _dependent_on_scheduled_merges(). svn path=/main/trunk/; revision=11011
* Bug #231382 - If the open() call in getconfig() raises an unexpected IOError,Zac Medico2008-07-101-0/+1
| | | | | | | write a message to stderr showing the exact call that failed, including the path of the file, in order to aid in debugging. svn path=/main/trunk/; revision=11010
* Make digestcheck() skip the "checking ebuild checksums" part when fetching inZac Medico2008-07-101-1/+1
| | | | | | the background. svn path=/main/trunk/; revision=11009
* * Fix AsynchronousTask._wait_hook() to prevent potential recursive callsZac Medico2008-07-101-2/+22
| | | | | | | | | | | triggered when calling exit listeners. * Make SequentialTaskQueue use exit listeners to trigger scheduling as soon as a task exits. This makes it possible for the queues to automatically schedule new tasks while scheduler remains in it's poll() loop. svn path=/main/trunk/; revision=11008
* * Handle errno.ECHILD and errno.ESRCH from waitpid() and kill() calls. TheseZac Medico2008-07-101-4/+28
| | | | | | | | | errors commonly occur when ^C is used to kill emerge. * Remove spawned pids from portage.process.spawned_pids immediately in order to avoid a race condition for removing it. svn path=/main/trunk/; revision=11007
* Fix Scheduler._is_restart_necessary() to return False if the replacementZac Medico2008-07-101-0/+48
| | | | | | | version of portage is the same exact version as the currently running instance. svn path=/main/trunk/; revision=11006
* Like in Binpkg, cancel or wait for the prefetcher in EbuildBuild.Zac Medico2008-07-091-4/+36
| | | | svn path=/main/trunk/; revision=11005
* Bug #231358 - Fix 'NameError: global name 'p' is not defined' inZac Medico2008-07-091-1/+1
| | | | | | getmaskingstatus() that's triggered by a conditional in LICENSE. svn path=/main/trunk/; revision=11004
* For parallel-fetch prefetchers, pass fetchonly=1 into the EbuildFetcherZac Medico2008-07-091-1/+2
| | | | | | constructor. svn path=/main/trunk/; revision=11003
* Use the EbuildFetcher class to implement async fetch for the EbuildBuildZac Medico2008-07-092-32/+48
| | | | | | | | | | | class. When in background mode, fetch output is logged to emerge-fetch.log. This solves a problem with calling doebuild(returnpid=1) and then doebuild calling fetch() synchronously. In that case, doebuild would return an int if fetch() failed (including when running pkg_nofetch), which would lead to a TypeError since doebuild(returnpid=1) is supposed to return a list containing a spawned pid. svn path=/main/trunk/; revision=11002
* * Unify the scheduler's "build" and "extract" queues into a singleZac Medico2008-07-091-11/+34
| | | | | | | | | | | | | | | "jobs" queue. * Add support for logging fetches to /var/log/emerge-fetch.log when --jobs is enabled. Previously this log was only used for the parallel-fetch feature but now it's also used for --jobs. The scheduler's "prefetch" queue has been renamed to "fetch" since it's not exclusively used for parallel-fetch anymore. * Pass the "background" parameter from Binpkg in to the BinpkgFetcher instance, to send output to emerge-fetch.log instead of stdout. svn path=/main/trunk/; revision=10999
* Fix SubProcess._poll to handle a missing pid gracefully.Zac Medico2008-07-091-0/+2
| | | | svn path=/main/trunk/; revision=10998
* Fix Scheduler._choose_pkg() to perform uninstallation actions (due toZac Medico2008-07-091-5/+25
| | | | | | blockers) in the correct order when building in parallel. svn path=/main/trunk/; revision=10996
* Fix _prune_digraph() to correctly distinguish "nomerge" nodes fromZac Medico2008-07-091-1/+2
| | | | | | "uninstall" nodes. svn path=/main/trunk/; revision=10995
* * Fix _choose_pkg() to pop the first node if there are no other jobs running.Zac Medico2008-07-081-2/+6
| | | | | | | | | * Allow _choose_pkg() to choose an uninstall node when it reaches the front of the queue. TODO: fix order for uninstall operations. * Fix _dependent_on_scheduled_merges() to handle direct circular deps correctly. svn path=/main/trunk/; revision=10994
* Make the "tight loop" assertion trigger whenever there is no state changeZac Medico2008-07-081-1/+1
| | | | | | detected for a given _schedule_main() call. svn path=/main/trunk/; revision=10993
* In scheduler._schedule_main(), only consider a poll() event to be a stateZac Medico2008-07-081-1/+1
| | | | | | change if it results in a handler being unregistered. svn path=/main/trunk/; revision=10992
* Fix Scheduler._dependent_on_scheduled_merges() to properly distinguishZac Medico2008-07-081-1/+1
| | | | | | "nomerge" nodes from "uninstall" nodes. svn path=/main/trunk/; revision=10991
* Remove redundant call to _schedule_main().Zac Medico2008-07-081-2/+0
| | | | svn path=/main/trunk/; revision=10990
* * Call self._wait() when unregistering output handlers, in order to avoidZac Medico2008-07-081-8/+24
| | | | | | | | | | | | triggering a tight loop. Also fix Subprocess._wait() to only schedule when self.registered is True. * Add an assertion inside Scheduler._schedule_main() to try and detect tight loops like the one above. * Fix typo in Scheduler._merge_exit(). svn path=/main/trunk/; revision=10989
* Derive PackageMerge from AsynchronousTask instead of CompositeTask sinceZac Medico2008-07-081-1/+1
| | | | | | CompositeTask is designed for self._current_task to be set. svn path=/main/trunk/; revision=10988
* Raise an AssertionError in CompositeTask._wait() if it's detected thatZac Medico2008-07-081-1/+7
| | | | | | | self._current_task hasn't been properly updated after calling wait on it. svn path=/main/trunk/; revision=10987
* Fix infinite loop in CompositeTask._wait().Zac Medico2008-07-081-1/+4
| | | | svn path=/main/trunk/; revision=10986
* Fix ValueError triggered when Scheduler._choose_pkg() doesn't find a packageZac Medico2008-07-081-1/+2
| | | | | | and returns None. svn path=/main/trunk/; revision=10985
* Use a normal list instead of a deque for Scheduler._pkg_queue since dequeZac Medico2008-07-081-3/+3
| | | | | | doesn't have a remove() method until python-2.5. svn path=/main/trunk/; revision=10984
* Implement parallel build support by adding new --jobs and --load-averageZac Medico2008-07-081-18/+238
| | | | | | | | options that are analogous to the corresponding `make` options. Input and output handling still need work to make it look better and act more friendly for things like interactive ebuilds that require input. svn path=/main/trunk/; revision=10983
* * Add background support to the SpawProcess class.Zac Medico2008-07-081-5/+10
| | | | | | * Make Binpkg pass the current background setting into subtasks. svn path=/main/trunk/; revision=10982
* Add AsynchronousTask.background attribute and add support for it to theZac Medico2008-07-081-15/+21
| | | | | | EbuildPhase class. svn path=/main/trunk/; revision=10981
* Also add "background" attributes to the EbuildBuild and Binpkg classes.Zac Medico2008-07-081-3/+5
| | | | svn path=/main/trunk/; revision=10980
* Add a MergeListItem.background attribute for putting a task in the backgroundZac Medico2008-07-081-4/+6
| | | | | | when max_jobs > 1. svn path=/main/trunk/; revision=10979
* Add a missing check in Scheduler._main_loop() to ensure that max_jobsZac Medico2008-07-081-0/+5
| | | | | | isn't exceeded. svn path=/main/trunk/; revision=10978
* Use plain sequential search for the SRC_URI.mirror check since the regexZac Medico2008-07-081-5/+8
| | | | | | approach is actually much slower. svn path=/main/trunk/; revision=10977
* Add a Scheduler._digraph attribute for use in parallel build scheduling.Zac Medico2008-07-071-0/+10
| | | | svn path=/main/trunk/; revision=10976
* Rename SubProcess.reg_id to _reg_id since it no longer needs to be exposed.Zac Medico2008-07-071-4/+5
| | | | svn path=/main/trunk/; revision=10975
* * Add new CompositeTask._default_final_exit() method and use it as the newZac Medico2008-07-071-7/+12
| | | | | | | | generic task exit callback. * Remove erroneous self.wait() calls inside MergeListItem.start(). svn path=/main/trunk/; revision=10974
* Add a new "SRC_URI.mirror" check which checks to if a uri listed inZac Medico2008-07-072-0/+22
| | | | | | | profiles/thirdpartymirrors is found in SRC_URI. Thanks to Betelgeuse for the initial patch which has now been optimized with a regex. svn path=/main/trunk/; revision=10973
* Move the "registered" attribute to the SubProcess class too.Zac Medico2008-07-071-3/+3
| | | | svn path=/main/trunk/; revision=10971
* Make SubProcess._wait() call scheduler.schedule(self.reg_id) so it'sZac Medico2008-07-071-6/+5
| | | | | | encapsulated and callers don't have to know about it. svn path=/main/trunk/; revision=10970
* remove TODO list as it's not maintained anywayMarius Mauch2008-07-071-10/+0
| | | | svn path=/main/trunk/; revision=10969
* * Implement CompositeTask._poll().Zac Medico2008-07-071-40/+147
| | | | | | | | | * Make AsynchronousTask classes call self.wait() to notify exit listeners. * Rewrite Scheduler._main_loop() to bring it closer to allowing parallel build scheduling. svn path=/main/trunk/; revision=10966
generated by cgit v1.2.3-1-g7c22 (git 2.25.1) at 2024-07-11 08:02:42 +0000