Where does SIOCGSTAMP come from?

I have an application that makes an ioctl call with SIOCGSTAMP to get
the time that a frame was received. This all works the way it's
supposed to. The problem is that the time on the system isn't accurate
enough for our application. What I'm looking for is in which part of
the code the timestamp is being saved. My plan is to change this
driver/library/part of the kernel so that it grabs the time off our
timing card instead.

Any pointers for where I should begin my search?

On May 22, 7:58*am, MegaGreg <gregorydav...@gmail.com> wrote:
> I have an application that makes an ioctl call with SIOCGSTAMP to get
> the time that a frame was received. This all works the way it's
> supposed to. The problem is that the time on the system isn't accurate
> enough for our application. What I'm looking for is in which part of
> the code the timestamp is being saved. My plan is to change this
> driver/library/part of the kernel so that it grabs the time off our
> timing card instead.
>
> Any pointers for where I should begin my search?

You are talking about the time a packet was received, right? If you're
talking gigabit ethernet, then it makes no sense to try to time it to
a higher degree of accuracy than a billionth of a second, since that
will just be the phase of the network card's clock.

Modern CPUs have a clock cycle counter, and a CPU that's over a
gigahertz already has a timebase more accurate than that. Any attempt
to access an external device would be so slow that it would only
reduce timestamp accuracy.

If you have an external timebase, use it to calibrate the TSC.

DS

> On May 22, 7:58 am, MegaGreg <gregorydav...@gmail.com> wrote:
>> I have an application that makes an ioctl call with SIOCGSTAMP to get
>> the time that a frame was received. This all works the way it's
>> supposed to. The problem is that the time on the system isn't accurate
>> enough for our application. What I'm looking for is in which part of
>> the code the timestamp is being saved. My plan is to change this
>> driver/library/part of the kernel so that it grabs the time off our
>> timing card instead.
>>
>> Any pointers for where I should begin my search?

It's been a while, but this used to happen just after the packet was
handed up from the hardware dependent layer.

Using a different clock may not solve your problem, depending on how
much precision you want. A decent high-speed driver will coalesce
interrupts, meaning you will get multiple packets handed over
simultaneously. If this matters, the only things you can do are (1)
find hardware (e.g. SysKonnect, I think) that can provide a timestamp
for you or (2) turn off interrupt coalescing at the expense of
performance (including lost packets).

The network traffic research folks have worried about this for years.

Davis Schwartz wrote:
> Modern CPUs have a clock cycle counter, and a CPU that's over a
> gigahertz already has a timebase more accurate than that. Any attempt
> to access an external device would be so slow that it would only
> reduce timestamp accuracy.

If the OP's problem is maintaining consistency between different
monitoring sites, then the problem is mapping the TSC into something
that can meaningfully be used for comparisons. The TSC wanders
slightly, depending on the ambient temperature. The raw TSC is fine for
short-term single-system comparisons, but it's not adequate for
long-term inter-system comparisons. NTP with a GPS PPS signal gets you
time-of-day to sub-millisecond accuracy if the TSC is reasonably well
behaved. Accurate hardware with a ToD clock that can be read via a
memory access is a much better solution. Any memory latency is down in
the noise compared to cumulative between-system TSC jitter/drift.

Thanks for the responses guys. I have more info interspersed.

On May 24, 1:00*pm, Allen McIntosh <nos...@mouse-potato.com> wrote:
> It's been a while, but this used to happen just after the packet was
> handed up from the hardware dependent layer.

This is what I figured, but I haven't been able to find where it
happens. I'll double check the ATM card driver. I haven't been able to
find the point at which the Linux ATM library calls a function for a
specific card. I think this may be where I need to be looking.

> Using a different clock may not solve your problem, depending on how
> much precision you want. *A decent high-speed driver will coalesce
> interrupts, meaning you will get multiple packets handed over
> simultaneously. *If this matters, the only things you can do are (1)
> find hardware (e.g. SysKonnect, I think) that can provide a timestamp
> for you or (2) turn off interrupt coalescing at the expense of
> performance (including lost packets).

I've considered the interrupt coalescing problem, but our timestamping
was always good enough before, so I'll leave it until it becomes a
problem.

> If the OP's problem is maintaining consistency between different
> monitoring sites, then the problem is mapping the TSC into something
> that can meaningfully be used for comparisons. *

This is exactly it, we need absolute time. We were using ntpd to
synchronise the clock, and were regularly getting 30-50us accuracy.
The problem is that it has to sync for about 8 hours first. We'll be
using a GPS timing card, which will be synced in about the time it
takes to boot, but that means we need to fiddle with the drivers,
getting the time from the card, rather than wherever it's getting it
from now. The acquisitioin time for the card is about 1us, which is
perfectly acceptable for our spec of +/-500us.

This brins me back around to the original problem. I've looked in the
ATM card driver (forerunnerHE), in the ATM library, and in the kernel
code, and I can't find the connections between them, where one part
calls to the other. Is there another section of code that I don't know
about?

I found what I was looking for. There is a call to __net_timestamp()
from he_service_rbrq() in drivers/atm/he.c This is the call that will
be replaced with a call to the timing card API. I'll have to wait
until the timing card comes in to see if it works. Thanks for the help.

On May 24, 9:00*am, Allen McIntosh <nos...@mouse-potato.com> wrote:

> If the OP's problem is maintaining consistency between different
> monitoring sites, then the problem is mapping the TSC into something
> that can meaningfully be used for comparisons. *The TSC wanders
> slightly, depending on the ambient temperature. *The raw TSC is fine for
> short-term single-system comparisons, but it's not adequate for
> long-term inter-system comparisons. *NTP with a GPS PPS signal gets you
> time-of-day to sub-millisecond accuracy if the TSC is reasonably well
> behaved. *Accurate hardware with a ToD clock that can be read via a
> memory access is a much better solution. *Any memory latency is down in
> the noise compared to cumulative between-system TSC jitter/drift.

It's not difficult at all to calibrate the TSC to an external source.
Every second, grab the TSC and grab the clock's value from your
reference source. Then simply linearly extrapolate a TSC value to get
the real time.

This will tend to be much more accurate than attempting to read the
external clock because there is no way to average out the jitter in
accessing the external clock when an event occurs. That jitter can
only be averaged out over several readings, as would occur when
calibrating the TSC.

DS

MegaGreg <(E-Mail Removed)> wrote:
> I have an application that makes an ioctl call with SIOCGSTAMP to
> get the time that a frame was received. This all works the way it's
> supposed to. The problem is that the time on the system isn't
> accurate enough for our application. What I'm looking for is in
> which part of the code the timestamp is being saved. My plan is to
> change this driver/library/part of the kernel so that it grabs the
> time off our timing card instead.

The NTP demi-gods in comp.protocols.time.ntp will no doubt find errors
in what I'm saying, but is the problem really the accuracy - ie is the
time the correct time - or is the problem the precision - ie is time
reported to enough significant digits for your purpose?

If the OS in question here is "linux" - you don't say what it is so
I'm guessing - you really should say with which OS... IIRC currently
the timestamps are taken in the driver once the driver is made aware
of the packet. Depending on your desires/requirements at that point
we have to start wondering about things like NIC interrupt coalescing
settings and what they mean for when the packet was on the wire vs
when the driver in the OS was made aware of the presence of the packet
in an inbound buffer.

> Any pointers for where I should begin my search?

If this is in the context of Linux, you should search the linux
"netdev" mailing list archives. You might also (linux or no) consider
the tcpdump mailing list archives.

rick jones
--
the road to hell is paved with business decisions...
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Rick Jones <(E-Mail Removed)> wrote:
> If the OS in question here is "linux" - you don't say what it is so
> I'm guessing - you really should say with which OS...

<insert emily litella reference here as rick notices the newsgroup >

rick jones
--
I don't interest myself in "why". I think more often in terms of
"when", sometimes "where"; always "how much." - Joubert
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...