Recording IQ-stream with rtlsdr and sdr# in the same format?

[Andreas Hornig]

Hi there,

I am new to this list and want to say hello and also ask my first question.

How do I set up the recording modes in RTLSDR software for linux and in
sdr# for windows in the same way?
I record the IQ-stream (baseband in sdr#) as 16 bit with my windows OS and
I asked a friend of mine to use rtlsdr and also record the iq-stream in
16bit. But his file size is only half that of mine.
I want to do auto cross-correlation and both streams need to be in the same
format to be comparable.

So I am not sure, if I only record with half of the bit-depth, or only one
channel with full bit-depth, or if something was set up wrongly with the
sample rate.

That are the CMDs we used

rtl_sdr -f 107.7e6 -s 2048000 -g 36.4 output.bin

sox -t raw -b 16 -e signed-integer -r 2048000 -B -c2 output.bin
output_2channel.wav

So, what shall I change in rtlsdr? In sdr# I can only select the bit-depth,
so I think I have to set rtlsdr in a different way, but how?

Thank you in advance! :)


Andreas

[Christophe Cattelain]

Hi,

if I am not mistaken, rtl_sdr(1) outputs 8 bits I/Q's (direct from the
dongle).

You should (at least) use 'sox -t raw -b 8 -c 2' to describe the input
format and Sox could probably transform it in signed 16 bits (little or
big endians) WAV file (?).

But I am not sure the rtl_sdr(1) output is 'signed char', as, in

Each I and Q value varies from 0 to 255, which represents
a range from -1 to +1. To get from the
unsigned (0-255) range you therefore subtract 127
(or 128 or 127.5) from each I and Q, giving you
a range from -127 to +128 (or -128 to +127, or -127.5 to +127.5)..

(ll140 in
https://github.com/bovine/dump1090-robb/blob/master/dump1090.c )

So, maybe, you'll need a small program to get the 8 bits right before
using sox(1) (?).

xof
==========================================

Hi there,
I am new to this list and want to say hello and also ask my first question.
How do I set up the recording modes in RTLSDR software for linux and in
sdr# for windows in the same way?
I record the IQ-stream (baseband in sdr#) as 16 bit with my windows OS
and I asked a friend of mine to use rtlsdr and also record the iq-stream
in 16bit. But his file size is only half that of mine.
I want to do auto cross-correlation and both streams need to be in the
same format to be comparable.
So I am not sure, if I only record with half of the bit-depth, or only
one channel with full bit-depth, or if something was set up wrongly with
the sample rate.
That are the CMDs we used
rtl_sdr -f 107.7e6 -s 2048000 -g 36.4 output.bin
sox -t raw -b 16 -e signed-integer -r 2048000 -B -c2 output.bin
output_2channel.wav
So, what shall I change in rtlsdr? In sdr# I can only select the
bit-depth, so I think I have to set rtlsdr in a different way, but how?
Thank you in advance! :)
Andreas

[Andreas Hornig]

Hi Christophe,

thank you for your reply

Post by Christophe Cattelain
Hi,
if I am not mistaken, rtl_sdr(1) outputs 8 bits I/Q's (direct from the
dongle).

That is also what I know, but I cannot find a good source for it :).

Post by Christophe Cattelain
You should (at least) use 'sox -t raw -b 8 -c 2' to describe the input
format and Sox could probably transform it in signed 16 bits (little or
big endians) WAV file (?).
But I am not sure the rtl_sdr(1) output is 'signed char', as, in

For me, it is important to have the same output format. If this means, I
have to set sdr# to 8bit and get that as a wav, and I could then also set
rtlsdr on linux to the same setting with two 8 bit streams for IQ, that
only needs to be converted to .wav, that would be okay.
But for me it is not yet clear, what both software do. Because we have set
both to 16bit and 2MHz sampling. But rtlsdr is half the size of that from
sdr#. So either 16bit for rtlsdr means, it is two channels with 8 bit each
and two channels with 16 bit each for sdr#, or there is something else.
So this is what I want to know before I decide about the converting with
sox.

Do you know how both software do their recordings with regars to bit depth
and channels?

Best regards,


Andreas

[Christophe Cattelain]

Post by Andreas Hornig
But for me it is not yet clear, what both software do. Because we have
set both to 16bit and 2MHz sampling. But rtlsdr is half the size of that
from sdr#. So either 16bit for rtlsdr means, it is two channels with 8
bit each and two channels with 16 bit each for sdr#, or there is
something else.

I don't know for SDR# but the rtl_sdr version I use outputs 8 bits I/Q's

Post by Andreas Hornig
while read(buf)
write(buf);

So, you get 16 bits per sample : 8 'I' bits and 8 'Q' bits (I am not
sure which comes first...). You can check this by recording one (or
ten) second(s) @ 2 MHz. You'll get a 4 (40) megabytes file.

Now, there is the question of what's in those 8 bits...

It seems that the rtl_sdr/dongle's output *is not* (signed char)
https://en.wikipedia.org/wiki/Two%27s_complement
but a 'shifted' unsigned char value (x+127 or 128).

You can see the twist in an old (?) GNU_Radio module
Loading Image...
(in http://sdr.osmocom.org/trac/wiki/rtl-sdr)
(I think that there is a single module that inputs rtl_sdr I/Q's and
outputs floats now (or that GNURadio rtl_sdr driver outputs complex floats)

It is probably not easy to spot it in the raw output but if you
transform I/Q in phase/magnitude you'll be able to plot the wave and
check it is ok.

I don't know if there is something in GNU Octave to read rtl_sdr format
but it would be usefull. (Octave has a wavread() function; it would be
nice to have a tutorial on how to read rtl_sdr I/Q's (from .wav or raw),
transform the I/Q stream in phase/magnitude, filter,...; in one word,
make something usefull with the data)


xof

[Peter Stuge]

Post by Christophe Cattelain
I don't know if there is something in GNU Octave to read rtl_sdr format
but it would be usefull. (Octave has a wavread() function; it would be
nice to have a tutorial on how to read rtl_sdr I/Q's (from .wav or raw),

Why not add the couple of lines needed in rtlsdr to support wav output,
if that's a common and useful format?


//Peter

[Andreas Hornig]

Hi there,

I just managed to make rtl_sdr work on my windows system, so I don't need
sdr# anymore. So it should be the same raw output on linux and on windows.
But still I need to convert it from raw to wav, because I want to use it
with python and numpy that offers wav support. What I would need then is
how to convert it right. When it is not signed integer, what do I have to
set in sox to create the wav? Can anyone state what rtl_sdr does with the
adc values and how to handle it?

Post by Peter Stuge
Why not add the couple of lines needed in rtlsdr to support wav output,
if that's a common and useful format?

That would be cool :). Then I could do the IQ processing with the python
code we currently code :).

Best regards,


Andreas

[Andreas Hornig]

Hi psb,

Hi!

Can anyone state what rtl_sdr does with the adc values and how to handle

it?
rtl_sdr does nothing with adc values - they are unsigned 8 bit integer. To
normalize them to -1..+1 range you need to subtract 128 (i.e. got signed
value) and divide that by 128.0.
(byte - 128) / 128.0
you can multiply result by 32767, round it and got indeed values which
sdr# store in their wav file.

firstly, I need to convert it to wav to be able to work with it, so I need
to know first, what settings I have to do in sox before I can use the math
and manipulate it :).
So it would be good to get the sox command to convert it right.

Best regards,

Andreas

[Peter Stuge]

Post by Andreas Hornig

rtl_sdr does nothing with adc values - they are unsigned 8 bit integer. To
normalize them to -1..+1 range you need to subtract 128 (i.e. got signed
value) and divide that by 128.0.
(byte - 128) / 128.0
you can multiply result by 32767, round it and got indeed values which
sdr# store in their wav file.

firstly, I need to convert it to wav to be able to work with it,

The samples are bytes in the file. It's a lot easier to operate on a
.raw file than to deal with a .wav, if you're just writing a small
program yourself.

Post by Andreas Hornig
So it would be good to get the sox command to convert it right.

Did you look at the sox man page? Maybe this works:

sox -r 2000k -e unsigned -b 8 -c 2 input.raw output.wav

But I think you should send a patch to output .wav directly instead.


//Peter

[Andreas Hornig]

Hi Peter,

Post by Peter Stuge
The samples are bytes in the file. It's a lot easier to operate on a
.raw file than to deal with a .wav, if you're just writing a small
program yourself.

For the .raw file, I couldn't find if there is a header or where the adc
bytes start. And we already started with .wav, because these were produced
by sdr# and we can already process the samples in .wav. And we can use
numpy and other standard features without building our own stuff.

Post by Peter Stuge
sox -r 2000k -e unsigned -b 8 -c 2 input.raw output.wav

As said I can convert it, but I was unsure about the -e parameter. So if it
is unsigned, it is great to know this. The offset calculation is rather
easy to do. I just wanted to be sure not to change the inputs by accident
by selecting the wrong -e :).

Post by Peter Stuge
But I think you should send a patch to output .wav directly instead.

If you mean me to add this, I am sorry that it is out of my scope and I
would prefer someone else to add this nice feature :).

So I will test what you all had said to me so far and I hope it will work
as intended.

Best regards,


Andreas

[Skip Tavakkolian]

i'm here to learn; it would be great if someone could clear up some
confusion for me:

if i understand my i/q for dummies[1] correctly, rtlsdr's output is
2,000,000/sec I/Q readings, each representing a single measurement of the
signal at (5E-7)th of a second (i.e. 1/2E6). i've understood the .wav file
to be a linear PCM sampling of audio at 44100 samples/second (one sample
per 1/44100 th of a second). if i did my math correctly you would have to
do *something* with around 45 I/Q samples to convert them to one .wav
sample and this assumes that I/Q represents sampling of an audio signal.
is this right?

[1] http://whiteboard.ping.se/SDR/IQ

thanks,
-Skip

On Fri, Oct 10, 2014 at 11:01 AM, Andreas Hornig <

Post by Andreas Hornig
Hi Peter,

Post by Peter Stuge
The samples are bytes in the file. It's a lot easier to operate on a
.raw file than to deal with a .wav, if you're just writing a small
program yourself.

For the .raw file, I couldn't find if there is a header or where the adc
bytes start. And we already started with .wav, because these were produced
by sdr# and we can already process the samples in .wav. And we can use
numpy and other standard features without building our own stuff.

Post by Peter Stuge
sox -r 2000k -e unsigned -b 8 -c 2 input.raw output.wav

As said I can convert it, but I was unsure about the -e parameter. So if
it is unsigned, it is great to know this. The offset calculation is rather
easy to do. I just wanted to be sure not to change the inputs by accident
by selecting the wrong -e :).

Post by Peter Stuge
But I think you should send a patch to output .wav directly instead.

If you mean me to add this, I am sorry that it is out of my scope and I
would prefer someone else to add this nice feature :).
So I will test what you all had said to me so far and I hope it will work
as intended.
Best regards,
Andreas

[psb hlw]

In general, you are right - you have to do *something* with all these data :)
For example, after you got I/Q from receiver (2M smps/sec) you can do frequency shift, filtering, demodulate, then filtering demod'd data with decimating (your 45 smps into one output sample).

To be even more clear, mentioned .wav with I/Q data in this thread (which sdr# stores) is RAW I/Q data but just with wav header. This is NOT demodulated/audio data. It contains sample_rate=2000000 bytes/sec in header.

if i understand my i/q for dummies[1] correctly, rtlsdr's output is 2,000,000/sec I/Q readings, each representing a single measurement of the signal at (5E-7)th of a second (i.e. 1/2E6). i've understood the .wav file to be a linear PCM sampling of audio at 44100 samples/second (one sample per 1/44100 th of a second). if i did my math correctly you would have to do *something* with around 45 I/Q samples to convert them to one .wav sample and this assumes that I/Q represents sampling of an audio signal. is this right?
[1] http://whiteboard.ping.se/SDR/IQ
thanks,
-Skip
Hi Peter,
The samples are bytes in the file. It's a lot easier to operate on a
.raw file than to deal with a .wav, if you're just writing a small
program yourself.
For the .raw file, I couldn't find if there is a header or where the adc bytes start. And we already started with .wav, because these were produced by sdr# and we can already process the samples in .wav. And we can use numpy and other standard features without building our own stuff.
sox -r 2000k -e unsigned -b 8 -c 2 input.raw output.wav
As said I can convert it, but I was unsure about the -e parameter. So if it is unsigned, it is great to know this. The offset calculation is rather easy to do. I just wanted to be sure not to change the inputs by accident by selecting the wrong -e :).
But I think you should send a patch to output .wav directly instead.
If you mean me to add this, I am sorry that it is out of my scope and I would prefer someone else to add this nice feature :).
So I will test what you all had said to me so far and I hope it will work as intended.
Best regards,
Andreas

[Skip Tavakkolian]

cool! that's very informative.

one possible solution to Andreas' problem might be to copy the raw samples
from the sdr# .wav file to stdout -- which is then theoretically comparable
to a .raw output. if so, here's a Go program that can be used:

https://gist.github.com/3195a2e68b412ca40927.git

i just modified an example that comes with this WAV file handling package
https://github.com/cryptix/wav

-Skip

Post by psb hlw
In general, you are right - you have to do *something* with all these data :)
For example, after you got I/Q from receiver (2M smps/sec) you can do
frequency shift, filtering, demodulate, then filtering demod'd data with
decimating (your 45 smps into one output sample).
To be even more clear, mentioned .wav with I/Q data in this thread (which
sdr# stores) is RAW I/Q data but just with wav header. This is NOT
demodulated/audio data. It contains sample_rate=2000000 bytes/sec in header.
if i understand my i/q for dummies[1] correctly, rtlsdr's output is
2,000,000/sec I/Q readings, each representing a single measurement of the
signal at (5E-7)th of a second (i.e. 1/2E6). i've understood the .wav file
to be a linear PCM sampling of audio at 44100 samples/second (one sample
per 1/44100 th of a second). if i did my math correctly you would have to
do *something* with around 45 I/Q samples to convert them to one .wav
sample and this assumes that I/Q represents sampling of an audio signal.
is this right?
[1] http://whiteboard.ping.se/SDR/IQ
thanks,
-Skip
On Fri, Oct 10, 2014 at 11:01 AM, Andreas Hornig <

Post by Andreas Hornig
Hi Peter,

Post by Peter Stuge
The samples are bytes in the file. It's a lot easier to operate on a
.raw file than to deal with a .wav, if you're just writing a small
program yourself.

For the .raw file, I couldn't find if there is a header or where the
adc bytes start. And we already started with .wav, because these were
produced by sdr# and we can already process the samples in .wav. And we can
use numpy and other standard features without building our own stuff.

Post by Peter Stuge
sox -r 2000k -e unsigned -b 8 -c 2 input.raw output.wav

As said I can convert it, but I was unsure about the -e parameter. So
if it is unsigned, it is great to know this. The offset calculation is
rather easy to do. I just wanted to be sure not to change the inputs by
accident by selecting the wrong -e :).

Post by Peter Stuge
But I think you should send a patch to output .wav directly instead.

If you mean me to add this, I am sorry that it is out of my scope and I
would prefer someone else to add this nice feature :).
So I will test what you all had said to me so far and I hope it will work as intended.
Best regards,
Andreas

[Skip Tavakkolian]

here's the right link to the gist file:

https://gist.github.com/9nut/3195a2e68b412ca40927


On Fri, Oct 10, 2014 at 4:04 PM, Skip Tavakkolian <

Post by Skip Tavakkolian
cool! that's very informative.
one possible solution to Andreas' problem might be to copy the raw samples
from the sdr# .wav file to stdout -- which is then theoretically comparable
https://gist.github.com/3195a2e68b412ca40927.git
i just modified an example that comes with this WAV file handling package
https://github.com/cryptix/wav
-Skip

Post by psb hlw
In general, you are right - you have to do *something* with all these data :)
For example, after you got I/Q from receiver (2M smps/sec) you can do
frequency shift, filtering, demodulate, then filtering demod'd data with
decimating (your 45 smps into one output sample).
To be even more clear, mentioned .wav with I/Q data in this thread (which
sdr# stores) is RAW I/Q data but just with wav header. This is NOT
demodulated/audio data. It contains sample_rate=2000000 bytes/sec in header.
if i understand my i/q for dummies[1] correctly, rtlsdr's output is
2,000,000/sec I/Q readings, each representing a single measurement of the
signal at (5E-7)th of a second (i.e. 1/2E6). i've understood the .wav file
to be a linear PCM sampling of audio at 44100 samples/second (one sample
per 1/44100 th of a second). if i did my math correctly you would have to
do *something* with around 45 I/Q samples to convert them to one .wav
sample and this assumes that I/Q represents sampling of an audio signal.
is this right?
[1] http://whiteboard.ping.se/SDR/IQ
thanks,
-Skip
On Fri, Oct 10, 2014 at 11:01 AM, Andreas Hornig <

Post by Andreas Hornig
Hi Peter,

Post by Peter Stuge
The samples are bytes in the file. It's a lot easier to operate on a
.raw file than to deal with a .wav, if you're just writing a small
program yourself.

For the .raw file, I couldn't find if there is a header or where the
adc bytes start. And we already started with .wav, because these were
produced by sdr# and we can already process the samples in .wav. And we can
use numpy and other standard features without building our own stuff.

Post by Peter Stuge
sox -r 2000k -e unsigned -b 8 -c 2 input.raw output.wav

As said I can convert it, but I was unsure about the -e parameter. So
if it is unsigned, it is great to know this. The offset calculation is
rather easy to do. I just wanted to be sure not to change the inputs by
accident by selecting the wrong -e :).

Post by Peter Stuge
But I think you should send a patch to output .wav directly instead.

If you mean me to add this, I am sorry that it is out of my scope and
I would prefer someone else to add this nice feature :).
So I will test what you all had said to me so far and I hope it will work as intended.
Best regards,
Andreas

[Andreas Hornig]

Hi,

for me it is, that I need it as wav to be able to use the pretty features
in numpy :).
But besides that, wav is only the sample container for it, but one numpy
directly understands :).

Best regards,

Andreas

[Peter Stuge]

Post by Andreas Hornig

Post by Peter Stuge
sox -r 2000k -e unsigned -b 8 -c 2 input.raw output.wav

As said I can convert it, but I was unsure about the -e parameter.

What the bits mean was described previously in this thread. From the
sox man-page unsigned might not be correct but is the best fit.

Post by Andreas Hornig

Post by Peter Stuge
But I think you should send a patch to output .wav directly instead.

If you mean me to add this, I am sorry that it is out of my scope and I
would prefer someone else to add this nice feature :).

Of course you would, but since you are the one who wants/needs it
right now, it seems that you're the one to do it. If you don't that's
of course fine too, but I think it would be a nice contribution that
others would appreciate.

Post by Andreas Hornig
So I will test what you all had said to me so far and I hope it will
work as intended.

Good luck!


//Peter

[Thierry Leconte]

rtl dongle output a 8bit unsigned interger for both I/Q signals.
To convert it to float man must remove the zero value.
A signed 8bit is normally between -128 and 127 so the convertion must
be something like :
I=(float)rtlinbuff[i++]-128.0; /* with rtlinbuff the input 8bit
unsigned buffer */
Q=(float)rtlinbuff[i++]-128.0; /* and I/Q float values */

But, I try to find that 0 level by a very simple low pass filter.
something like :
IM=0.99999*IM+0.00001*(double)rtlinbuff[i++];
QM=0.99999*QM+0.00001*(double)rtlinbuff[i++];

and after a few seconds of running with antenna disconnected , I find a
value around 127.35 for a first dongle and 127.4 for a second.
I do some differents runs and these values seem consistant.

So,
1) is it something true with other dongles ?
2) Is it documented somewhere and I just discover what everybody knows ?
3) It will be interesting to modify rtl_test to show this value if it
vary from one dongle to another.

[psb hlw]

I'm sure, the zero will be on different places on different dongles. You don't need to know where it is exactly (i.e., some constant). Subtracting 128 is enough for further processing.

Post by Thierry Leconte
rtl dongle output a 8bit unsigned interger for both I/Q signals.
To convert it to float man must remove the zero value.
I=(float)rtlinbuff[i++]-128.0; /* with rtlinbuff the input 8bit unsigned buffer */
Q=(float)rtlinbuff[i++]-128.0; /* and I/Q float values */
IM=0.99999*IM+0.00001*(double)rtlinbuff[i++];
QM=0.99999*QM+0.00001*(double)rtlinbuff[i++];
and after a few seconds of running with antenna disconnected , I find a value around 127.35 for a first dongle and 127.4 for a second.
I do some differents runs and these values seem consistant.
So,
1) is it something true with other dongles ?
2) Is it documented somewhere and I just discover what everybody knows ?
3) It will be interesting to modify rtl_test to show this value if it vary from one dongle to another.

[Thierry Leconte]

Post by psb hlw
I'm sure, the zero will be on different places on different dongles. You don't need to know where it is exactly (i.e., some constant). Subtracting 128 is enough for further processing.

I suspect that in fact 127 is the true good value plus some small DC
offset that effectively must depend of the dongle.

[Douglas Hall]

It seems like the majority of these dongles have an offset​ similar to what
you found. Mine is right at about 127.4 like you found.