PTIMER: Timer engine

Introduction

PTIMER is a small functional unit used to measure time by the card. It has a 56-bit tick counter connected to a programmable clock source. The current value of this counter is used for timestamping by many other units on the GPU. Two such timestamps can be substracted to get the wall time elapsed between their creation and measure eg. command execution time. Also, it’s possible to set up an interrupt that will be triggered when the low 27 bits of the counter reach a specified value.

The PTIMER’s MMIO range is 0x101000:0x102000 on NV1, 0x9000:0xa000 on NV3 and later cards. It is enabled by PMC.ENABLE bit 4 [NV1 - shared with PDMA] or 16 [NV3-], and its interrupt line is connected to PMC.INTR line 20. It’s available on all cards.

Curiously, on NV41+ the PTIMER is also used to report MMIO faults, ie. MMIO space accesses from host that failed for some reason.

Todo

document that some day].

MMIO register list - NV1

8-bit space nv1-ptimer [0x1000]
nv1-mmio 0x101000: PTIMER
Address Name Description
0x100 INTR interrupt status/acknowledge
0x140 INTR_ENABLE interrupt enable
0x200 CLOCK_DIV clock divider
0x210 CLOCK_MUL clock multiplier
0x400 TIME_LOW low part of the time counter
0x404 TIME_HIGH high part of the time counter
0x410 ALARM the TIME_LOW value to interrupt on

MMIO register list - NV3-

8-bit space nv3-ptimer [0x1000]
nv3-mmio 0x9000: PTIMER
g80-mmio 0x9000: PTIMER
gf100-mmio 0x9000: PTIMER
Address Variants Name Description
0x60 G80: ??? ???
0x64 G80: ??? ???
0x80 NV17:NV20,NV25:G80 ??? ???
0x80 GF100: ??? ???
0x84 NV41: MMIO_FAULT_ADDR ???
0x88 NV41: MMIO_FAULT_DATA ???
0x100 all INTR interrupt status/acknowledge
0x140 all INTR_ENABLE interrupt enable
0x200 all CLOCK_DIV clock divider
0x210 all CLOCK_MUL clock multiplier
0x220 NV41: CLOCK_SOURCE clock source selection
0x400 all TIME_LOW low part of the time counter
0x410 all TIME_HIGH high part of the time counter
0x420 all ALARM the TIME_LOW value to interrupt on

The clock source

The clock that PTIMER counts is generated by applying a selectable ratio to a clock source. The clock source depends on the card:

On NV41+ cards, which have both internal and external clock generators, the internal clock generator and the switch is configured by the CLOCK_SOURCE register:

reg32 ptimer-clock-source
nv3-ptimer 0x220: CLOCK_SOURCE [NV41:]
  • bits 0-7: INTERNAL_MUL - specifies the multiplier of internal clock generator minus 1
  • bits 8-11: INTERNAL_DIV - specifies the divisor of internal clock generator minus 1
  • bit 16: SELECT - if 0, internal clock source used, if 1 external source used

The internal clock generator will generate a clock with frequency given by crystal_frequency * (MUL + 1) / (DIV + 1). However, it is not a PLL, but a simple counter - it cannot generate a clock of a higher frequency than what PTIMER logic itself is clocked at, which is equal to the external clock.

The clock ratio

The clock source is frequency-converted by a simple counter-based converter before being used for counting. The converter multiplies the frequency by the specified ratio. The registers are:

reg32 ptimer-clock-div
nv1-ptimer 0x200: CLOCK_DIV
nv3-ptimer 0x200: CLOCK_DIV
  • bits 0-15: clock divider - should not be 0
reg32 ptimer-clock-mul
nv1-ptimer 0x210: CLOCK_MUL
nv3-ptimer 0x210: CLOCK_MUL
  • bits 0-15: clock multiplier - has to be between 0 and the clock divider, 0 stops the counter entirely

The clock used for the counter is clock_source * CLOCK_MUL / CLOCK_DIV. It’s not possible to get a higher frequency than the clock source - the converter will misbehave.

The time counter

PTIMER’s clock is a 56-bit value that is spread across two 32-bit registers:

reg32 ptimer-time-low
nv1-ptimer 0x400: TIME_LOW
nv3-ptimer 0x400: TIME_LOW
  • bits 5-31: low 27 bits of the counter
  • bits 0-4: always 0
reg32 ptimer-time-high
nv1-ptimer 0x404: TIME_HIGH
nv3-ptimer 0x410: TIME_HIGH
  • bits 0-28: high 29 bits of the counter
  • bits 29-31: always 0

The counter is thus embedded in bits 5-60 of a 64-bit number split across the two 32-bit words. Whenever the PTIMER clock is requested by other parts of the card, the returned timestamp will be this 64-bit number. Because of the 5-bit shift, the timestamps are actually counted in units of 1/32 of PTIMER tick, with resolution of 32 ticks.

Also, TIME_LOW bit 17 [ie. bit 12 of the actual counter] is connected to a PCOUNTER signal on NV10:GF100, called PTIMER_TIME_B12.

Reading the clock

In order to accurately read the clock, the following code should be used:

uint32 high1, high2, low;

do
{
        high1 = mmio_rd32(TIME_HIGH);
        low = mmio_rd32(TIME_LOW);
        high2 = mmio_rd32(TIME_HIGH);
} while (high1 != high2);

This code works around the “mutual dependency”. No matter in what order the registers are read, an issue may arise and lead to an error of 2^32 as show by the following examples:

  • TIME_LOW is read, overflows and then TIME_HIGH is read
  • TIME_HIGH is read, TIME_LOW overflows, TIME_LOW is read

The proposed code checks no overflow on TIME_LOW happened between the moment we read TIME_HIGH and the moment we read TIME_HIGH again. If it happened, we start again until it succeeds.

The alarm and interrupts

PTIMER can also be used to trigger an interrupt when TIME_LOW matches a specified value. The registers dealing with interrupts are:

reg32 ptimer-intr
nv1-ptimer 0x100: INTR
nv3-ptimer 0x100: INTR

Status of interrupts generated by PTIMER. On read, returns 1 for bits corresponding to pending interrupts. On write, if 1 is written to a bit, its interrupt gets cleared, if 0 is written nothing happens.

reg32 ptimer-intr-enable
nv1-ptimer 0x140: INTR_ENABLE
nv3-ptimer 0x140: INTR_ENABLE

Interrupt enable bitmask. Set to enable, clear to disable. Interrupts that are masked will still show up in INTR when they’re triggered, but won’t cause the PTIMER interrupt line to go active.

The bitfields common to these registers are:

  • bit 0: ALARM - triggered whenever value of ALARM register is equal to value of TIME_LOW register

The alarm time is set in:

reg32 ptimer-alarm
nv1-ptimer 0x410: ALARM
nv3-ptimer 0x420: ALARM
  • bits 5-31: alarm time - when this equals the value of bits 5-31 of TIME_LOW, the ALARM interrupt will be triggered
  • bits 0-4: always 0

Unknown registers

Todo

figure these out

reg32 ptimer-unk060
nv3-ptimer 0x60: ??? [G80:]

???

reg32 ptimer-unk064
nv3-ptimer 0x64: ??? [G80:]

???

reg32 ptimer-unk080-nv17
nv3-ptimer 0x80: ??? [NV17:NV20,NV25:G80]

???

reg32 ptimer-unk080-gf100
nv3-ptimer 0x80: ??? [GF100:]

???

reg32 ptimer-mmio-fault-data
nv3-ptimer 0x88: MMIO_FAULT_DATA [NV41:]

Todo

write me

reg32 ptimer-mmio-fault-addr
nv3-ptimer 0x84: MMIO_FAULT_ADDR [NV41:]

Todo

write me

Todo

document MMIO_FAULT_*