Re: GSoC: Add SMT/HT awareness to DragonFlyBSD scheduler

[Mihai Carabas <mihai.carabas@xxxxxxxxx>]

--20cf30363aed53e31e04c18f549e
Content-Type: text/plain; charset=ISO-8859-1

Hello,

This week, I managed to *dynamicly detect *the CPU topology of a
platform. There
are 2 components: platform depedent code which determines the mask_size
within APICID for logical and core ids. For now, is available only for
64bit, intel and amd (I will make code available for 32bit soon). The code
was written here [1], from line 1107.

Another component, is the platform independent. I used the logical_CPU_bits
and core_bits to determine the number of entities per level and build a
topology tree (I tried to make all as general as possible). For this code I
added a new file [2]. The init function is registered through SYSINIT to
run before the initialization of the USCHED. All the topology information
is available through sysctl.

Here are outputs for 3 different topologies. Unfortunately, I don't have an
AMD platform to test on. All tests are done on INTEL platforms.

*1) two physical CPUs (emulated through VMWARE - yes vmware gives this
posibility)*

hw.cpu_topology.tree:
\-PACKAGE MEMBERS: cpu0 cpu1
  |-CHIP ID 0: cpu0
  \-CHIP ID 1: cpu1
hw.cpu_topology.members: cpu0 cpu1

hw.cpu_topology.cpu0.physical_id: 0
hw.cpu_topology.cpu0.physical_sibings: cpu0

hw.cpu_topology.cpu1.physical_id: 1
hw.cpu_topology.cpu1.physical_sibings: cpu1


*2) two cores on one physical CPU*

hw.cpu_topology.tree:
\-PACKAGE MEMBERS: cpu0 cpu1
  \-CHIP ID 0: cpu0 cpu1
    |-CORE ID 0: cpu0
    \-CORE ID 1: cpu1
hw.cpu_topology.members: cpu0 cpu1

hw.cpu_topology.cpu0.physical_id: 0
hw.cpu_topology.cpu0.physical_sibings: cpu0 cpu1
hw.cpu_topology.cpu0.core_id: 0
hw.cpu_topology.cpu0.core_siblings: cpu0

hw.cpu_topology.cpu1.physical_id: 0
hw.cpu_topology.cpu1.physical_sibings: cpu0 cpu1
hw.cpu_topology.cpu1.core_id: 1
hw.cpu_topology.cpu1.core_siblings: cpu1


*3) two cores on one physical CPU and two threads on each core.*

hw.cpu_topology.tree:
\-PACKAGE MEMBERS: cpu0 cpu1 cpu2 cpu3
  \-CHIP ID 0: cpu0 cpu1 cpu2 cpu3
    |-CORE ID 0: cpu0 cpu2
    | |-THREAD ID 0: cpu0
    | \-THREAD ID 1: cpu2
    \-CORE ID 1: cpu1 cpu3
      |-THREAD ID 0: cpu1
      \-THREAD ID 1: cpu3
hw.cpu_topology.members: cpu0 cpu1 cpu2 cpu3

hw.cpu_topology.cpu0.physical_id: 0
hw.cpu_topology.cpu0.physical_sibings: cpu0 cpu1 cpu2 cpu3
hw.cpu_topology.cpu0.core_id: 0
hw.cpu_topology.cpu0.core_siblings: cpu0 cpu2

hw.cpu_topology.cpu1.physical_id: 0
hw.cpu_topology.cpu1.physical_sibings: cpu0 cpu1 cpu2 cpu3
hw.cpu_topology.cpu1.core_id: 1
hw.cpu_topology.cpu1.core_siblings: cpu1 cpu3

hw.cpu_topology.cpu2.physical_id: 0
hw.cpu_topology.cpu2.physical_sibings: cpu0 cpu1 cpu2 cpu3
hw.cpu_topology.cpu2.core_id: 0
hw.cpu_topology.cpu2.core_siblings: cpu0 cpu2

hw.cpu_topology.cpu3.physical_id: 0
hw.cpu_topology.cpu3.physical_sibings: cpu0 cpu1 cpu2 cpu3
hw.cpu_topology.cpu3.core_id: 1
hw.cpu_topology.cpu3.core_siblings: cpu1 cpu3

Now, every CPU, have in its own structure, the knowledge of the whole
topology. The next step is to develop some meaningful tests and begin
playing with scheduling decisions. Scheduling will take into account the
load of the target's siblings cpu, too.

For those who want to test this, they have to clone the repo [3]. May be
next week, if you want, I will make a full patch with my code till now.

Any feedback is welcome!

Thanks,
Mihai Carabas

[1]
https://github.com/mihaicarabas/dragonfly/blob/master/sys/platform/pc64/x86_64/mp_machdep.c
[2]
https://github.com/mihaicarabas/dragonfly/blob/master/sys/kern/cpu_topology.c
[3] https://github.com/mihaicarabas/dragonfly

--20cf30363aed53e31e04c18f549e
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello,<div><br></div><div>This week, I managed to <i style=3D"font-weight:b=
old">dynamicly detect </i>the CPU topology of a platform.=A0<span style>The=
re are 2 components: platform depedent code which determines the mask_size =
within APICID for logical and core ids. For now, is available only for 64bi=
t, intel and amd (I will make code available for 32bit soon). The code was =
written here [1], from line 1107.</span></div>
<div><div style><br></div><div style>Another component, is the platform ind=
ependent. I used the logical_CPU_bits and core_bits to determine the number=
 of entities per level and build a topology tree (I tried to make all as ge=
neral as possible). For this code I added a new file [2]. The init function=
 is registered through SYSINIT to run before the initialization of the USCH=
ED.=A0All the topology information is available through sysctl.</div>
<div style><br></div><div style>Here are outputs for 3 different topologies=
. Unfortunately, I don&#39;t have an AMD platform to test on. All tests are=
 done on INTEL platforms.</div><div style><br></div><div style><b><i>1) two=
 physical CPUs (emulated through VMWARE - yes vmware gives this posibility)=
</i></b></div>
<div style><div><br></div><div>hw.cpu_topology.tree:</div><div>\-PACKAGE ME=
MBERS: cpu0 cpu1</div><div>=A0 |-CHIP ID 0: cpu0</div><div>=A0 \-CHIP ID 1:=
 cpu1</div><div>hw.cpu_topology.members: cpu0 cpu1</div><div><br></div><div=
>
hw.cpu_topology.cpu0.physical_id: 0</div><div>hw.cpu_topology.cpu0.physical=
_sibings: cpu0</div><div><br></div><div>hw.cpu_topology.cpu1.physical_id: 1=
</div><div>hw.cpu_topology.cpu1.physical_sibings: cpu1</div><div><br></div>
</div><div style><br></div><div style><b><i>2) two cores on one physical CP=
U</i></b></div><div style><div><br></div><div>hw.cpu_topology.tree:</div><d=
iv>\-PACKAGE MEMBERS: cpu0 cpu1</div><div>=A0 \-CHIP ID 0: cpu0 cpu1</div>
<div>=A0 =A0 |-CORE ID 0: cpu0</div><div>=A0 =A0 \-CORE ID 1: cpu1</div><di=
v>hw.cpu_topology.members: cpu0 cpu1</div><div><br></div><div>hw.cpu_topolo=
gy.cpu0.physical_id: 0</div><div>hw.cpu_topology.cpu0.physical_sibings: cpu=
0 cpu1</div>
<div>hw.cpu_topology.cpu0.core_id: 0</div><div>hw.cpu_topology.cpu0.core_si=
blings: cpu0</div><div><br></div><div>hw.cpu_topology.cpu1.physical_id: 0</=
div><div>hw.cpu_topology.cpu1.physical_sibings: cpu0 cpu1</div><div>hw.cpu_=
topology.cpu1.core_id: 1</div>
<div>hw.cpu_topology.cpu1.core_siblings: cpu1</div></div><div style><br></d=
iv><div style><br></div><div style><i><b>3) two cores on one physical CPU a=
nd two threads on each core.</b></i></div><div style><div><br></div><div>
hw.cpu_topology.tree:</div><div>\-PACKAGE MEMBERS: cpu0 cpu1 cpu2 cpu3</div=
><div>=A0 \-CHIP ID 0: cpu0 cpu1 cpu2 cpu3</div><div>=A0 =A0 |-CORE ID 0: c=
pu0 cpu2</div><div>=A0 =A0 | |-THREAD ID 0: cpu0</div><div>=A0 =A0 | \-THRE=
AD ID 1: cpu2</div>
<div>=A0 =A0 \-CORE ID 1: cpu1 cpu3</div><div>=A0 =A0 =A0 |-THREAD ID 0: cp=
u1</div><div>=A0 =A0 =A0 \-THREAD ID 1: cpu3</div><div>hw.cpu_topology.memb=
ers: cpu0 cpu1 cpu2 cpu3</div><div><br></div><div>hw.cpu_topology.cpu0.phys=
ical_id: 0</div>
<div>hw.cpu_topology.cpu0.physical_sibings: cpu0 cpu1 cpu2 cpu3</div><div>h=
w.cpu_topology.cpu0.core_id: 0</div><div>hw.cpu_topology.cpu0.core_siblings=
: cpu0 cpu2</div><div><br></div><div>hw.cpu_topology.cpu1.physical_id: 0</d=
iv>
<div>hw.cpu_topology.cpu1.physical_sibings: cpu0 cpu1 cpu2 cpu3</div><div>h=
w.cpu_topology.cpu1.core_id: 1</div><div>hw.cpu_topology.cpu1.core_siblings=
: cpu1 cpu3</div><div><br></div><div>hw.cpu_topology.cpu2.physical_id: 0</d=
iv>
<div>hw.cpu_topology.cpu2.physical_sibings: cpu0 cpu1 cpu2 cpu3</div><div>h=
w.cpu_topology.cpu2.core_id: 0</div><div>hw.cpu_topology.cpu2.core_siblings=
: cpu0 cpu2</div><div><br></div><div>hw.cpu_topology.cpu3.physical_id: 0</d=
iv>
<div>hw.cpu_topology.cpu3.physical_sibings: cpu0 cpu1 cpu2 cpu3</div><div>h=
w.cpu_topology.cpu3.core_id: 1</div><div>hw.cpu_topology.cpu3.core_siblings=
: cpu1 cpu3</div></div><div style><br></div><div style>Now, every CPU, have=
 in its own structure, the knowledge of the whole topology. The next step i=
s to develop some meaningful tests and begin playing with scheduling decisi=
ons. Scheduling will take into account the load of the target&#39;s sibling=
s cpu, too.</div>
<div class=3D"gmail_quote"><br></div></div><div class=3D"gmail_quote">For t=
hose who want to test this, they have to clone the repo [3]. May be next we=
ek, if you want, I will make a full patch with my code till now.</div><div =
class=3D"gmail_quote">
<div style><br></div><div style>Any feedback is welcome!</div><div style><b=
r></div><div style>Thanks,</div><div style>Mihai Carabas</div><div style><b=
r></div><div style>[1]=A0<a href=3D"https://github.com/mihaicarabas/dragonf=
ly/blob/master/sys/platform/pc64/x86_64/mp_machdep.c" target=3D"_blank" sty=
le=3D"color:rgb(17,85,204)">https://github.com/mihaicarabas/dragonfly/blob/=
master/sys/platform/pc64/x86_64/mp_machdep.c</a></div>
<div style>[2]=A0<a href=3D"https://github.com/mihaicarabas/dragonfly/blob/=
master/sys/kern/cpu_topology.c" target=3D"_blank" style=3D"color:rgb(17,85,=
204)">https://github.com/mihaicarabas/dragonfly/blob/master/sys/kern/cpu_to=
pology.c</a></div>
</div><div class=3D"gmail_quote">[3]=A0<a href=3D"https://github.com/mihaic=
arabas/dragonfly">https://github.com/mihaicarabas/dragonfly</a></div>

--20cf30363aed53e31e04c18f549e--