Re: Question about userland scheduler

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:Hello,
:
:Matt, could you please explain yet another time how usched interacts with
:LWKT scheduler. I've read your posts in kernel@ from Sat, 15 Nov 2003 and 
:Tue, 7 Dec 2004 and still don't get it :(
:
:>   The userland scheduler is responsible ONLY for scheduling threads that
:>   are running in userland. The userland scheduler schedules only one such
:>   thread at a time, and always at e.g. TDPRI_USER_NORM.  It does not 
:>   schedule multiple threads (running in userland) with LWKT at the same 
:>   time. 
:
:What means 'only one such thread at a time' and how about other userland
:threads?
:
:In the situation when we have to schedule TDPRI_USER_NORM thread
:(useland thread) it will be choosen by LWKT scheduler according to round-
:robin discipline. So, why we need the userland scheduler in this situation?
:
:I know that I'm wrong, so please show me the situation where the
:userland scheduler is used and explain how it is used on top of LWKT
:scheduler.
:
:And the final question for now: what is gd->gd_uschedcp? Is this a pointer
:to the current running process?
:
:Thanks.
:
:-- 
:Sergey Glushchenko

    The LWKT scheduler is a fixed priority scheduler.  This is suitable for
    well prioritized kernel threads but not really suitable for userland
    threads.  The userland scheduler is responsible for scheduling processes
    running in or about to return to user mode.  It does this by scheduling
    the process's thread via the LWKT scheduler at near the lowest LWKT priority
    (so all kernel threads will have priority over any user process thread
    running in userland).  But since the userland scheduler only has a handful
    (3) of LWKT priorities to play with it can really only LWKT-schedule one
    runnable user process at a time (per cpu) to get the desired behavior.

    The biggest issue with the userland scheduler right now is the fact that
    the userland process run queue and related structures are global rather
    then per-cpu.  I need to rewrite the whole thing, actually, and I intend to
    after the release to finally make that path MP safe.  I would prefer to 
    make the userland scheduler per-cpu, with process threads clearly assigned
    to a particular cpu, and then have the per-cpu schedulers proactively 'push'
    processes to other cpus based on a heuristic.  This will not only give us
    excellent locality of reference, it will also allow me to implement a 
    secondary 'desired' cpu for userland processes so we don't bounce them across
    all the cpus on a SMP system (blowing up all the caches in the process).

    Theoretically we could make the userland scheduler LWKT-schedule all runnable
    threads and then bump the LWKT priority of the one it really wants to run
    the most.  In fact, I like this idea better then the current implementation.
    It wasn't done this way initially because the current implementation uses a
    global mutex (the Big Giant Lock) and 'pulls' runnable processes out of 
    the user scheduler's run queue onto an available cpu and a process's thread
    can't be pulled to another cpu if it's already scheduled.  When I rewrite 
    the scheduler into a push system it will be able to just leave all the 
    user process threads LWKT scheduled and simply fiddle with the LWKT priority.

					-Matt
					Matthew Dillon 
					<dillon@xxxxxxxxxxxxx>