Author Topic: Data consumption  (Read 9143 times)

exfirepro

Re: Data consumption
« Reply #15 on: February 24, 2021, 04:08:11 pm »
Hi Vince,

Like Alan, I also don’t want to put anyone to extra trouble, but without a baseline for average data use, I fear that many just won’t be prepared take on the burden of paying for a 4/5G Network Service to set up an ATOM station in rural areas. I see this being a major drawback to expanding the network to fill the gaps in the remoter areas of Scotland, Wales and the North of England, away from the major centres of population.

Hence I repeat the earlier plea to anyone currently running an ATOM station on a mobile network (or otherwise able to determine actual data usage on their wired, cable or fibre network) to get in touch and help us collate some figures on data use.

Best Regards

Peter

Vince

Re: Data consumption
« Reply #16 on: February 24, 2021, 11:07:15 pm »
Measuring accurately the data usage has always been something I have wanted to do but just never got round to it. My firewall (expensive NGFW type) allows me to set a policy on a device to manage / monitor the traffic. Unfortunately without dumping the traffic logs for this policy off to a syslog system for later analysis, I can only see details of the last 24 hours and a cumulative bandwidth used. The cumulative figure over the last 2 months is about 15GB but this is combined upload and download figure. For data usage we are really only concerned with the download, as upload is not counted.

So I’m happy to do this for my own benefit but will hope it will help others. Though the pilotaware team should be able to tell if any site is running on a mobile connection as they would have access to the external IP of each site.

Vince
Sign up to Stranded Flyer to assist your fellow aviators in those difficult situations.

www.strandedflyer.net

exfirepro

Re: Data consumption
« Reply #17 on: February 25, 2021, 08:58:33 am »
Hi Vince,

Thanks for that. Any and all help gratefully appreciated.

I have wider access to the network than is available to individual station maintainers, but access to the type of data we need is still above my pay grade. I will speak to Lee and Chris about this and see what can be done. In the meantime any figures which can be provided by individual station owners / maintainers will be gratefully received.

Best Regards

Peter

steveu

Re: Data consumption
« Reply #18 on: February 25, 2021, 06:05:00 pm »
Steve

I think you are missing the point of my post.
Despite the claims of the mobile network providers that they cover 99% of the country, I'm sure we've all experienced visiting places that get poor or no signal on one or more networks.   So Lee's comment about £16.00 for unlimited data is not always going to be correct, as PaulSS has experienced.  I was not referring to regional (or post code) differences in pricing by any particular network provider but that the best priced provider may not be available in some areas, due to cr@ppy signal.

Not wanting to overly labour the point but there are only four physical networks in the UK at the moment.

EE
Three
Vodafone
O2.

There are good deals directly or via MVNOs on all of them. It’s a competitive market.

EE – direct, if you commit to 24 months, £19 for 15GB

EE MVNO - £15 for 10GB  (Asda mobile)

Three – direct, £10 for 12GB, going up to £14 for 30GB on 24 months

Three MVO - £15 for 50GB (iD mobile)

Vodafone - £24 for 1GB going up to £36 for unlimited

Vodafone MVNO - £15 for 15GB going up to £20 for 45GB (Voxi)

O2 – £21 for 9GB

O2 MVNO - £10 for 9GB data, or £20 for 100GB. (Giffgaff)

So, five minutes finds you a good deal if you can only use one network, if there’s a choice, then it gets better. Half those prices, in general, for 2GB.

The big issue is not the cost, for a club, strip with more than one resident, or an airfield. It’s peanuts.  My club has a mobile broadband set up that serves two video cameras… it never runs out of data, and costs under £10.

The issue is the cost for one station maintainer with no financial assistance or sharing of the burden. £20/month is £240/year, that's the cost of BMAA membership and a year’s aircraft permit. I don't expect one individual to carry that.

So I agree with you there about the cost...

I’m curious about data traffic levels, whilst I can imagine a very large number of transactions, I can’t see the actual transactions themselves being very big. I’m guessing the updates are the big data overheads, as maybe the whole image has to be downloaded, unzipped then the current files need to be overwritten. This means a big download.

It would need some work, but if the stock Raspbian repository were to be replaced by a custom PAW one, then apt could run once every day, and perhaps this would allow just the changed files to be downloaded. Apt is currently not used in the ATOM GRID.

This is clueless guesswork in the dark, as I have no idea how the update actually works, and I don’t expect that info to be outside the development team.

So my question would be, if there was one monthly update to the firmware, how big would it be? 430MB? That's a huge dollop...

How big a proportion of the total monthly data overhead would this be? Nebulous question, as it depends on the flying weather. 18 months ago, IIRC someone said about 150MB traffic?

I don't expect this question to be answered, as it's confidential.  But if it gets read, that's enough for me.

Vince

Re: Data consumption
« Reply #19 on: February 25, 2021, 10:00:49 pm »
Long post, I have a couple of comments :)

Your cameras will consume very little download as they are mostly uploading possibly huge amounts but still won’t cost a great deal.

I was the one who saw 150MB per month but this was when it was just uploading Flarm traffic. I got away with a Three 200Mb SIM which had no cost as is/was free.

I do not expect the 360Radar MLAT traffic will use much download data as this is mostly upload therefore this will be little cost.

Updates will be mostly download so need to be considered.

VRS will no doubt download data on aircraft that it displays which could be large if utilised a lot.

What I suspect will be the biggest download will be the data for aircraft for the rebroadcast.

Just out of interest my firewall list 157GB used accessing VRS on my LAN over the last few months. If this was done remotely it would be upload.

Vince
Sign up to Stranded Flyer to assist your fellow aviators in those difficult situations.

www.strandedflyer.net

steveu

Re: Data consumption
« Reply #20 on: February 25, 2021, 10:13:17 pm »
Your cameras will consume very little download as they are mostly uploading possibly huge amounts but still won’t cost a great deal.

All the networks I've used count uploads as part of the data allowance, and this is the case with the cameras using O2 via Giffgaff.

Vince

Re: Data consumption
« Reply #21 on: February 25, 2021, 11:09:30 pm »
On the prepaid Three data SIM’s I’ve bee using uploads haven’t counted.

The free SIM they do, just add 2 quid to it and you get 200MB free per month.

They do a 24GB prepaid which lasts for 2 years or until you consume the 24GB. On my 3rd one of these and at 42 quid is only £1.75 per month or per 1GB

https://www.amazon.co.uk/Three-Mobile-Pay-Broadband-Data-Black/dp/B01M3VJ2B2/ref=sr_1_1_sspa?crid=302TIRY1QEXFS&dchild=1&keywords=three+24gb+data+sim&qid=1614294403&sprefix=Three+24gb%2Caps%2C160&sr=8-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUExTFMyTjVMQTY2MlczJmVuY3J5cHRlZElkPUEwNDIwOTg2MlI2UVJGVlNDMVZGTiZlbmNyeXB0ZWRBZElkPUEwMDQ5ODI3Mkw2NVFPTzE1VzNDSyZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=
Sign up to Stranded Flyer to assist your fellow aviators in those difficult situations.

www.strandedflyer.net

neilmurg

Re: Data consumption
« Reply #22 on: February 27, 2021, 10:14:29 pm »
I own the network my ATOM GRID works on, and I can monitor data from the fixed IP of the ATOM GRID to the router.
But, I'm on a steep leaning curve with Linux/Mint/Pi. I have spare Pi devices that would do the monitoring, or on the Linux Mint laptop.
Can anyone tell me what tools to load and what commands to run to set up the monitoring? I'm OK with the CLI, but I'd need simple instructions, or hand holding. I built a Pi webcam and got it running, and I'm building a Pi-hole (as examples of my competence/incompetence).
This issue is relevant to me as I have an ATOM GRID user with a limited bandwidth internet feed.
« Last Edit: February 27, 2021, 10:16:12 pm by neilmurg »

Vince

Re: Data consumption
« Reply #23 on: February 28, 2021, 12:16:53 am »
I’m not sure what you mean by “I own the network”? The pi is connected to the local area network, which is owned by the user.

To monitor the bandwidth used you either need software on the device that can track the packets in/out of the network interface or place something between the device and the Internet. This in-between could be the Router but mostly they have limited functionality and may not provide bandwidth on a per device basis.

Vince
Sign up to Stranded Flyer to assist your fellow aviators in those difficult situations.

www.strandedflyer.net

exfirepro

Re: Data consumption
« Reply #24 on: February 28, 2021, 12:18:47 pm »
Hi Neil,

That would be very useful, but above my pay grade (knowledge level) to be able to tell you how to do it. Lee / Ash Vinning (PAW Support) and James Rose (Aircrew) might be able to help, or of course SteveU? I will ask.

Best Regards

Peter


JCurtis

Re: Data consumption
« Reply #25 on: February 28, 2021, 04:41:32 pm »
Do you get SSH access onto your own OGN node?  If so, this is a quick and dirty method for getting some stats.

You should just be able to run ifconfig and the Pi will give you the network stats.  Naturally, this includes any local traffic including the SSH session, but it would provide a worst-case set of numbers.

The Pi I have displaying some CCTV for example...

Code: [Select]
$ ifconfig

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.10.61  netmask 255.255.255.0  broadcast 192.168.10.255
        inet6 2a0b:e541:30c:64:b6fc:adb5:8d58:32ad  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::9498:487b:aa50:8605  prefixlen 64  scopeid 0x20<link>
        ether b8:27:eb:31:b0:dc  txqueuelen 1000  (Ethernet)
  -->   RX packets 313753834  bytes 3917725983 (3.6 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
  -->   TX packets 166464981  bytes 2120530336 (1.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

This is my WiFi interface, which since the last reboot has transmitted 1.9Gb and received 3.6Gb.

You can also just view the stats direct with...

Code: [Select]
cat /sys/class/net/wlan0/statistics/tx_bytes
cat /sys/class/net/wlan0/statistics/rx_bytes

.. change wlan0 for, probably, eth0 if you use the wired interface.  Note this is in bytes, so divide by 1073741824 for Gb.

So reboot the OGN station, take some numbers at the end of each day, average them and you will have the worst case consumption.

There is probably some way of just getting the stats for the process running the OGN side, or perhaps the TCP port or remote OGN node address.  I don't tinker with the Pi often enough to know what is in the various images out there.

Anyway, just a thought.
Designer and maker of charge4.harkwood.co.uk, smart universal USB chargers designed for aviation.  USB Type-A and USB-C power without the RF interference. Approved for EASA installs under CS-STAN too.

steveu

Re: Data consumption
« Reply #26 on: February 28, 2021, 05:56:51 pm »
So, my attempt:

Code: [Select]
pi@ognpaw:~ $ uptime
17:46:05 up 10 days,  1:49,  1 user,  load average: 3.06, 3.10, 3.20

I got:

Code: [Select]
pi@ognpaw:~ $ cat /sys/class/net/eth0/statistics/tx_bytes                                                                                                                                 
2071504778

1.92GB sent

Code: [Select]
pi@ognpaw:~ $ cat /sys/class/net/eth0/statistics/rx_bytes
1304085003

1.21GB received.

Uptime is 10 days, from first code box, checked with the command top, but top not so terse as to be useful here.

Does this help?

Station is PWEgerton, near Ashford, Kent.

My lying/making it up guesstimate is for 10GB a month. Please check the location as it will be getting caned for high level ADS-B, as it is overflown for both Heathrow and Gatwick, and also sees Southend traffic.





neilmurg

Re: Data consumption
« Reply #27 on: February 28, 2021, 06:05:19 pm »
Do you get SSH access onto your own OGN node?  If so, this is a quick and dirty method for getting some stats.
.........
Ooh, I like that, I'll give it a try, thanks
Yup
 Same result as with steveu, ifconfig is less typing, still needed 'uptime' to work out / month figures
and yes, I can get SSH access to the node, that's a new one on me as well, which helped with the copy / paste of data, thanks!

Learning the Pi / Linux is one of my lockdown projects
  But I must stop telling people I'm 'wrestling with Python', I keep getting funny looks..
« Last Edit: February 28, 2021, 06:51:43 pm by neilmurg »

neilmurg

Re: Data consumption
« Reply #28 on: February 28, 2021, 06:06:46 pm »
So, my attempt:......
I like this too! I'll report back, thanks

PWMurg
Uptime / data
2021/02/28 18:12 9 days 4:55
Tx 2643809832 2.46Gb 8.01Gb / month
Rx 4234573418 3.94Gb 12.84Gb / month

Station based in Esher, overflown by Heathrow, and the H9 helicopter route, just inside the TMA
« Last Edit: February 28, 2021, 06:33:45 pm by neilmurg »

Admin

Re: Data consumption
« Reply #29 on: March 01, 2021, 07:55:04 am »
Unfortunately this method does not discriminate between data destined for LAN or WAN addresses on eth0
You have to run tcpdump on the eth0 interface, exclude traffic sent to the LAN, and count the data

Thx
Lee