PilotAware

British Forum => OGN-R PilotAware => Topic started by: Wadoadi on September 23, 2019, 12:02:48 am

Title: Data consumption
Post by: Wadoadi on September 23, 2019, 12:02:48 am
Hi
   Potentially looking at setting up a a ground station at our airfield, however we only have a 4G router and limited data allowance.

While I know the data usage will be variable, does anyone have some idea of the amount consumed per day?

Also is it permissible and possible to schedule the system to work during a given time period per day?

Thanks
Title: Re: Data consumption
Post by: Ian Melville on September 23, 2019, 06:37:35 am
I think someone asked and was given data usage figure awhile back. Don't have any figures myself.

I use a plug top mains timer to turn mine at home off each night and back on a few hours later. In effect doing a reboot each night. Mine is mechanical, but you would want a battery-powered or backed up one. if your electricity goes off at any time.
Title: Re: Data consumption
Post by: Admin on September 23, 2019, 07:48:50 am
Hi Adrian, Ian
I added an internal feature to throttle the amount of data transfer, I could probably make this user configurable
What kind of allowance are we talking, rate or data over a period?
Thx
Lee
Title: Re: Data consumption
Post by: exfirepro on September 23, 2019, 08:11:53 am
Hi Adi,

There is some info re 4G data usage from Vince in this thread... http://forum.pilotaware.com/index.php/topic,980.msg16470.html#msg16470 which may be useful, though this will of course vary with location and traffic.

Edit. Just saw Lee’s post above.

The ‘traditional’ setup for OGN-R Stations is to switch them off via a timer (originally mechanical) between about 11pm and 4am as the glider traffic they were originally designed to rebroadcast wouldn’t (normally) be flying during these hours. This daily shutdown performed two main functions. It ‘rested’ the units - potentially prolonging the life of components and more importantly, it forced a reboot in the early hours, as part of which the software checked for and installed any new updates which had been introduced since the last restart. In the case of 4G linked units of course, it also limits data use by preventing the transmission of data when it’s not required.

Things have moved on somewhat, however, with checks for software updates now taking place automatically throughout the day and not requiring a daily hard reboot to instigate their install. This spreads the load and means that stations update automatically almost as soon as updates become available. The addition of Mode-S/3D Rebroadcast also extends the likely operating hours for VFR Traffic, though this is still unlikely to be required (by VFR Users) outwith the 4am to 11pm operating window except perhaps at the height of Summer in the far north. I now run my stations on electronic timers, which I find much more reliable as they have built in battery backup, and so maintain ‘real time’ through any short term power cut or shut down (whereas the mechanical timers need resetting after every power-out) and I now try to run the stations as near as possible to normal daylight hours, though you could of course still use timing to help control data usage.

Hope this helps.

Regards

Peter
Title: Re: Data consumption
Post by: Wadoadi on September 23, 2019, 10:55:36 pm
Thanks All,
Lee, currently we have 15gb but we are getting through most of that! We are looking at getting a new sim and allowance which will give us either 45Gb or 100gb or 200gb depending on the option and costs we go for.

Our current speeds are around 10mbps up and 16mbps down but we do get higher sometimes!

I have not checked the latency.

We would like to re broadcast cast the ADSB/3D info also
Title: Re: Data consumption
Post by: PaulSS on February 21, 2021, 09:03:40 am
Yes, a bit of a Lazarus moment for this thread  ;D

Given the passage of time, has anyone got any updated information on the data used by an ATOM station? I see from the link in this thread that Vince's unit was using approximately 150MB per month but just wanted to check if this is likely to be higher as an ATOM station linked to 360Radar, as opposed to just an OGNR station. Mine is normally on 24/7 but I do have an electronic timer if necessary.

Our (potential) new house is going to require a 4G router and sim card, as the Internet down the pipes is appalling, so I just need to check what sort of data allowance I'm going to need in the future.
Title: Re: Data consumption
Post by: Admin on February 21, 2021, 10:07:17 am
Hi all

I am interested to know if concerns are related to data rate, or data consumption

I think when the thread began it was consumption, after a few weeks quick google I see you can now obtain unlimited data SIM cards for £16

I would be happy to try to thoroughly examine and optimise data consumption, but what are the goals and expectations?

Thx
Lee
Title: Re: Data consumption
Post by: PaulSS on February 21, 2021, 11:45:46 am
Hi Lee,

I'm just interested to see if it would really eat into a limited plan (Vodaphone is going to have to be our 'choice', whether we like it or not). If it's in the order of 300-400MB per month then I wouldn't even sneeze at it. If it's going to consume 100GB then I might have something more to say  ;D
Title: Re: Data consumption
Post by: exfirepro on February 21, 2021, 12:35:03 pm
Lee,

I've had quite a few discussions about this recently from existing site owners and in connection with trying to persuade 'remote' sites to install an Atom Station. Most of these sites are very small clubs, sometimes only 3 or 4 members or even single operator, and are therefore concerned at the potential cost implications.

Obviously it's difficult to be precise as data usage will be dependent on traffic, but one site for example (aware of the relatively high frequency of recent ATOM updates) has asked whether the system can be configured to 'advise' when updates are available but only install them when the maintainer has a chance to temporarily relocate the unit to a 'Home Base' with unlimited data access. (I said we wouldn't be keen on this as it impinges on our ability to make rapid changes and would inevitable lead to stations running different versions on an ad-hoc basis).

A better appreciation of potential data usage from say a more remote and a busy site would be extremely useful (not least when looking at potential '4/5G only' sites and trying to persuade the site owners to install them).

Best Regards

Peter
Title: Re: Data consumption
Post by: steveu on February 21, 2021, 02:21:03 pm
I'm just interested to see if it would really eat into a limited plan (Vodaphone is going to have to be our 'choice', whether we like it or not).

At the risk of being a nosy parker and therefore being ignored as is justified, may I venture the following?

I assume you have to take Vodafone for reasons of coverage? Have you done a full survey? Assuming yes. If so, would there be a better deal on an MVNO (Mobile Virtual Network Operator) using the Vodafone backbone?

Like this?

https://www.4g.co.uk/sim-only/voxi/ (https://www.4g.co.uk/sim-only/voxi/)

Title: Re: Data consumption
Post by: PaulSS on February 21, 2021, 04:08:05 pm
Hi Steve,

Thank you for your interest and ideas. Yes, I'll need to use Vodaphone because all the others don't necessarily provide decent coverage and the current owners of the house have done the same research and come to the same conclusions. There is a Vodaphone mast that seems to give pretty good coverage of the area and my idea is to stick an external antenna on one of the chimneys and point it at the mast. I should achieve line of sight by doing this and, with the Poynting 11dB gain antenna, I'm hoping this will ensure I get a decent signal to the router.

There is no 5G coverage at the moment but my thinking is to at least future-proof a bit by (a)buying the antenna mentioned above, which will be 5G capable and (b)buying a 5G capable router (hopefully). Until this point I didn't think it was worth paying the extra for a 5G data plan when I can only get 4G as a maximum but the plans you've suggested have an unlimited data 5G plan which is basically the same price as the Vodaphone unlimited 4G plan.

Having done more digging I am much, much less concerned (tending towards zero) about the ATOM station data use, as the unlimited plans are on a par with what I'm paying for Sky broadband at the moment, so I have no compunction in transferring to one of those. If the VOXI 5G plan only gives me 4G but a bit of future proofing then I think it's worth going for that, so thank you for your suggestion.
Title: Re: Data consumption
Post by: AlanG on February 23, 2021, 04:14:57 pm
Hi Guys
Looks like a positive result Paul but, even taking Lee's comments into consideration, the cynic in me says that not all areas, especially the more remote, are blessed with the best deals, so it would be good to hear from any station operator who is currently running an Atom station on a purely 4G router and can give a fairly accurate estimate of monthly data usage.  I realise that this may vary by situation depending on traffic volumes and running periods etc. but some indication could be extremely useful when trying to persuade potential operators for more remote sites.  Nobody wants the effort of setting up a site only to fall by the wayside because the running costs are unacceptable.

If anyone owns, or is aware of, a station operating on 4G perhaps they could point them in this direction.

Regards
Alan

Title: Re: Data consumption
Post by: steveu on February 23, 2021, 10:34:36 pm
Looks like a positive result Paul but, even taking Lee's comments into consideration, the cynic in me says that not all areas, especially the more remote, are blessed with the best deals

I don't understand this. I'm not aware of mobile phone or mobile broadband/data deals being post code or location limited so it's just a question of coverage, and if there is only one network at that location on 4G, then you either find a deal on that network or on an MVNO. If there's no or poor coverage, then cost doesn't come into it.

MVNOs don't have the same bells and whistles as the premium packages on physical networks, and tend to offer very good SIM only, 30 day notice to quit packages. You already have a router, so you don't want a long contract with new phone cost being recovered to tear you a new one. Picks - GiffGaff, an O2 MVNO, 9GB for £10/month, 20GB for £15, or 100GB for £20.

There's a good list of MVNOs here:

https://www.uswitch.com/mobiles/guides/mvnos-a-guide-to-coverage-and-pros-and-cons-of-virtual-networks/ (https://www.uswitch.com/mobiles/guides/mvnos-a-guide-to-coverage-and-pros-and-cons-of-virtual-networks/)

Paul mentioned external antennas, and they can make all the difference in poor coverage areas, sometimes meaning a jump from 3G to 4G. AS a result of the signal gains from external, optional antennas with my little wireless Huawei box, I bought a proper 4G mobile broadband router next time round with Ethernet connectors and sensible antennas.

Title: Re: Data consumption
Post by: Vince on February 24, 2021, 01:35:56 am
I originally had OGN-R at the airfield using a 4G router and Three prepaid 24GB SIM. This was originally just receiving PAW & FLARM so it only really uploaded and downloaded very little. As upload does not count towards the data allowance the pre-purchased 24GB was never consumed.

When the changes to OGN-R came in and it started to download traffic for rebroadcast this changed, the data started to be consumed quickly. This coincided with power issues at the airfield so the OGN-R was shutdown until we could sort the lack of power & lack of data. We have sorted the power out at the airfield now but my monitoring of the data used by ATOM at home has not been promising with large amounts download data.

I currently have a spare 4G router and access to a SIM with substantial data allowance so happy to run up my station off this for a few months. I can then post daily data usages that Vodafone record, which will be more accurate than my firewall provides.

Vince   

 
Title: Re: Data consumption
Post by: AlanG on February 24, 2021, 03:28:26 pm
Steve

I think you are missing the point of my post.
Despite the claims of the mobile network providers that they cover 99% of the country, I'm sure we've all experienced visiting places that get poor or no signal on one or more networks.   So Lee's comment about £16.00 for unlimited data is not always going to be correct, as PaulSS has experienced.  I was not referring to regional (or post code) differences in pricing by any particular network provider but that the best priced provider may not be available in some areas, due to cr@ppy signal.

Vince

Nice offer but would not want to put you to any extra effort for something that is mainly academic at the moment.
I believe some home routers are capable of reporting data usage of individual devices on the network but unfortunately my Virgin router is not that sophisticated.  I thought there may be some smaller strips that currently used, or had tried this method and found the costs prohibitive or otherwise.  :-\

Regards
Alan
Title: Re: Data consumption
Post by: exfirepro on February 24, 2021, 04:08:11 pm
Hi Vince,

Like Alan, I also don’t want to put anyone to extra trouble, but without a baseline for average data use, I fear that many just won’t be prepared take on the burden of paying for a 4/5G Network Service to set up an ATOM station in rural areas. I see this being a major drawback to expanding the network to fill the gaps in the remoter areas of Scotland, Wales and the North of England, away from the major centres of population.

Hence I repeat the earlier plea to anyone currently running an ATOM station on a mobile network (or otherwise able to determine actual data usage on their wired, cable or fibre network) to get in touch and help us collate some figures on data use.

Best Regards

Peter
Title: Re: Data consumption
Post by: Vince on February 24, 2021, 11:07:15 pm
Measuring accurately the data usage has always been something I have wanted to do but just never got round to it. My firewall (expensive NGFW type) allows me to set a policy on a device to manage / monitor the traffic. Unfortunately without dumping the traffic logs for this policy off to a syslog system for later analysis, I can only see details of the last 24 hours and a cumulative bandwidth used. The cumulative figure over the last 2 months is about 15GB but this is combined upload and download figure. For data usage we are really only concerned with the download, as upload is not counted.

So I’m happy to do this for my own benefit but will hope it will help others. Though the pilotaware team should be able to tell if any site is running on a mobile connection as they would have access to the external IP of each site.

Vince
Title: Re: Data consumption
Post by: exfirepro on February 25, 2021, 08:58:33 am
Hi Vince,

Thanks for that. Any and all help gratefully appreciated.

I have wider access to the network than is available to individual station maintainers, but access to the type of data we need is still above my pay grade. I will speak to Lee and Chris about this and see what can be done. In the meantime any figures which can be provided by individual station owners / maintainers will be gratefully received.

Best Regards

Peter
Title: Re: Data consumption
Post by: steveu on February 25, 2021, 06:05:00 pm
Steve

I think you are missing the point of my post.
Despite the claims of the mobile network providers that they cover 99% of the country, I'm sure we've all experienced visiting places that get poor or no signal on one or more networks.   So Lee's comment about £16.00 for unlimited data is not always going to be correct, as PaulSS has experienced.  I was not referring to regional (or post code) differences in pricing by any particular network provider but that the best priced provider may not be available in some areas, due to cr@ppy signal.

Not wanting to overly labour the point but there are only four physical networks in the UK at the moment.

EE
Three
Vodafone
O2.

There are good deals directly or via MVNOs on all of them. It’s a competitive market.

EE – direct, if you commit to 24 months, £19 for 15GB

EE MVNO - £15 for 10GB  (Asda mobile)

Three – direct, £10 for 12GB, going up to £14 for 30GB on 24 months

Three MVO - £15 for 50GB (iD mobile)

Vodafone - £24 for 1GB going up to £36 for unlimited

Vodafone MVNO - £15 for 15GB going up to £20 for 45GB (Voxi)

O2 – £21 for 9GB

O2 MVNO - £10 for 9GB data, or £20 for 100GB. (Giffgaff)

So, five minutes finds you a good deal if you can only use one network, if there’s a choice, then it gets better. Half those prices, in general, for 2GB.

The big issue is not the cost, for a club, strip with more than one resident, or an airfield. It’s peanuts.  My club has a mobile broadband set up that serves two video cameras… it never runs out of data, and costs under £10.

The issue is the cost for one station maintainer with no financial assistance or sharing of the burden. £20/month is £240/year, that's the cost of BMAA membership and a year’s aircraft permit. I don't expect one individual to carry that.

So I agree with you there about the cost...

I’m curious about data traffic levels, whilst I can imagine a very large number of transactions, I can’t see the actual transactions themselves being very big. I’m guessing the updates are the big data overheads, as maybe the whole image has to be downloaded, unzipped then the current files need to be overwritten. This means a big download.

It would need some work, but if the stock Raspbian repository were to be replaced by a custom PAW one, then apt could run once every day, and perhaps this would allow just the changed files to be downloaded. Apt is currently not used in the ATOM GRID.

This is clueless guesswork in the dark, as I have no idea how the update actually works, and I don’t expect that info to be outside the development team.

So my question would be, if there was one monthly update to the firmware, how big would it be? 430MB? That's a huge dollop...

How big a proportion of the total monthly data overhead would this be? Nebulous question, as it depends on the flying weather. 18 months ago, IIRC someone said about 150MB traffic?

I don't expect this question to be answered, as it's confidential.  But if it gets read, that's enough for me.
Title: Re: Data consumption
Post by: Vince on February 25, 2021, 10:00:49 pm
Long post, I have a couple of comments :)

Your cameras will consume very little download as they are mostly uploading possibly huge amounts but still won’t cost a great deal.

I was the one who saw 150MB per month but this was when it was just uploading Flarm traffic. I got away with a Three 200Mb SIM which had no cost as is/was free.

I do not expect the 360Radar MLAT traffic will use much download data as this is mostly upload therefore this will be little cost.

Updates will be mostly download so need to be considered.

VRS will no doubt download data on aircraft that it displays which could be large if utilised a lot.

What I suspect will be the biggest download will be the data for aircraft for the rebroadcast.

Just out of interest my firewall list 157GB used accessing VRS on my LAN over the last few months. If this was done remotely it would be upload.

Vince
Title: Re: Data consumption
Post by: steveu on February 25, 2021, 10:13:17 pm
Your cameras will consume very little download as they are mostly uploading possibly huge amounts but still won’t cost a great deal.

All the networks I've used count uploads as part of the data allowance, and this is the case with the cameras using O2 via Giffgaff.
Title: Re: Data consumption
Post by: Vince on February 25, 2021, 11:09:30 pm
On the prepaid Three data SIM’s I’ve bee using uploads haven’t counted.

The free SIM they do, just add 2 quid to it and you get 200MB free per month.

They do a 24GB prepaid which lasts for 2 years or until you consume the 24GB. On my 3rd one of these and at 42 quid is only £1.75 per month or per 1GB

https://www.amazon.co.uk/Three-Mobile-Pay-Broadband-Data-Black/dp/B01M3VJ2B2/ref=sr_1_1_sspa?crid=302TIRY1QEXFS&dchild=1&keywords=three+24gb+data+sim&qid=1614294403&sprefix=Three+24gb%2Caps%2C160&sr=8-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUExTFMyTjVMQTY2MlczJmVuY3J5cHRlZElkPUEwNDIwOTg2MlI2UVJGVlNDMVZGTiZlbmNyeXB0ZWRBZElkPUEwMDQ5ODI3Mkw2NVFPTzE1VzNDSyZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU= (https://www.amazon.co.uk/Three-Mobile-Pay-Broadband-Data-Black/dp/B01M3VJ2B2/ref=sr_1_1_sspa?crid=302TIRY1QEXFS&dchild=1&keywords=three+24gb+data+sim&qid=1614294403&sprefix=Three+24gb%2Caps%2C160&sr=8-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUExTFMyTjVMQTY2MlczJmVuY3J5cHRlZElkPUEwNDIwOTg2MlI2UVJGVlNDMVZGTiZlbmNyeXB0ZWRBZElkPUEwMDQ5ODI3Mkw2NVFPTzE1VzNDSyZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=)
Title: Re: Data consumption
Post by: neilmurg on February 27, 2021, 10:14:29 pm
I own the network my ATOM GRID works on, and I can monitor data from the fixed IP of the ATOM GRID to the router.
But, I'm on a steep leaning curve with Linux/Mint/Pi. I have spare Pi devices that would do the monitoring, or on the Linux Mint laptop.
Can anyone tell me what tools to load and what commands to run to set up the monitoring? I'm OK with the CLI, but I'd need simple instructions, or hand holding. I built a Pi webcam and got it running, and I'm building a Pi-hole (as examples of my competence/incompetence).
This issue is relevant to me as I have an ATOM GRID user with a limited bandwidth internet feed.
Title: Re: Data consumption
Post by: Vince on February 28, 2021, 12:16:53 am
I’m not sure what you mean by “I own the network”? The pi is connected to the local area network, which is owned by the user.

To monitor the bandwidth used you either need software on the device that can track the packets in/out of the network interface or place something between the device and the Internet. This in-between could be the Router but mostly they have limited functionality and may not provide bandwidth on a per device basis.

Vince
Title: Re: Data consumption
Post by: exfirepro on February 28, 2021, 12:18:47 pm
Hi Neil,

That would be very useful, but above my pay grade (knowledge level) to be able to tell you how to do it. Lee / Ash Vinning (PAW Support) and James Rose (Aircrew) might be able to help, or of course SteveU? I will ask.

Best Regards

Peter

Title: Re: Data consumption
Post by: JCurtis on February 28, 2021, 04:41:32 pm
Do you get SSH access onto your own OGN node?  If so, this is a quick and dirty method for getting some stats.

You should just be able to run ifconfig and the Pi will give you the network stats.  Naturally, this includes any local traffic including the SSH session, but it would provide a worst-case set of numbers.

The Pi I have displaying some CCTV for example...

Code: [Select]
$ ifconfig

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.10.61  netmask 255.255.255.0  broadcast 192.168.10.255
        inet6 2a0b:e541:30c:64:b6fc:adb5:8d58:32ad  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::9498:487b:aa50:8605  prefixlen 64  scopeid 0x20<link>
        ether b8:27:eb:31:b0:dc  txqueuelen 1000  (Ethernet)
  -->   RX packets 313753834  bytes 3917725983 (3.6 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
  -->   TX packets 166464981  bytes 2120530336 (1.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

This is my WiFi interface, which since the last reboot has transmitted 1.9Gb and received 3.6Gb.

You can also just view the stats direct with...

Code: [Select]
cat /sys/class/net/wlan0/statistics/tx_bytes
cat /sys/class/net/wlan0/statistics/rx_bytes

.. change wlan0 for, probably, eth0 if you use the wired interface.  Note this is in bytes, so divide by 1073741824 for Gb.

So reboot the OGN station, take some numbers at the end of each day, average them and you will have the worst case consumption.

There is probably some way of just getting the stats for the process running the OGN side, or perhaps the TCP port or remote OGN node address.  I don't tinker with the Pi often enough to know what is in the various images out there.

Anyway, just a thought.
Title: Re: Data consumption
Post by: steveu on February 28, 2021, 05:56:51 pm
So, my attempt:

Code: [Select]
pi@ognpaw:~ $ uptime
17:46:05 up 10 days,  1:49,  1 user,  load average: 3.06, 3.10, 3.20

I got:

Code: [Select]
pi@ognpaw:~ $ cat /sys/class/net/eth0/statistics/tx_bytes                                                                                                                                 
2071504778

1.92GB sent

Code: [Select]
pi@ognpaw:~ $ cat /sys/class/net/eth0/statistics/rx_bytes
1304085003

1.21GB received.

Uptime is 10 days, from first code box, checked with the command top, but top not so terse as to be useful here.

Does this help?

Station is PWEgerton, near Ashford, Kent.

My lying/making it up guesstimate is for 10GB a month. Please check the location as it will be getting caned for high level ADS-B, as it is overflown for both Heathrow and Gatwick, and also sees Southend traffic.




Title: Re: Data consumption
Post by: neilmurg on February 28, 2021, 06:05:19 pm
Do you get SSH access onto your own OGN node?  If so, this is a quick and dirty method for getting some stats.
.........
Ooh, I like that, I'll give it a try, thanks
Yup
 Same result as with steveu, ifconfig is less typing, still needed 'uptime' to work out / month figures
and yes, I can get SSH access to the node, that's a new one on me as well, which helped with the copy / paste of data, thanks!

Learning the Pi / Linux is one of my lockdown projects
  But I must stop telling people I'm 'wrestling with Python', I keep getting funny looks..
Title: Re: Data consumption
Post by: neilmurg on February 28, 2021, 06:06:46 pm
So, my attempt:......
I like this too! I'll report back, thanks

PWMurg
Uptime / data
2021/02/28 18:12 9 days 4:55
Tx 2643809832 2.46Gb 8.01Gb / month
Rx 4234573418 3.94Gb 12.84Gb / month

Station based in Esher, overflown by Heathrow, and the H9 helicopter route, just inside the TMA
Title: Re: Data consumption
Post by: Admin on March 01, 2021, 07:55:04 am
Unfortunately this method does not discriminate between data destined for LAN or WAN addresses on eth0
You have to run tcpdump on the eth0 interface, exclude traffic sent to the LAN, and count the data

Thx
Lee
Title: Re: Data consumption
Post by: JCurtis on March 01, 2021, 08:05:11 am
Unfortunately this method does not discriminate between data destined for LAN or WAN addresses on eth0
You have to run tcpdump on the eth0 interface, exclude traffic sent to the LAN, and count the data

Thx
Lee

Yes, I put that caveat in the above post. It should exclude any internal data via the loop back port though.  A worst case data set is probably better than no data or educated guesses.
Title: Re: Data consumption
Post by: neilmurg on March 01, 2021, 08:29:27 am
Challenge accepted
Title: Re: Data consumption
Post by: steveu on March 01, 2021, 10:55:09 am
Unfortunately this method does not discriminate between data destined for LAN or WAN addresses on eth0
You have to run tcpdump on the eth0 interface, exclude traffic sent to the LAN, and count the data

No one has been at the strip to fly, and neither the kiosk Pi nor access to the VRS has been used during lockdown, would you expect there to be a big difference between WAN/LAN in this case??

The LAN traffic would be a worse case scenario, with WAN traffic lower, so as a worst case, it covers the data requirement for a mobile broadband connection.

According to which and dpkg, tcpdump is not installed, so at that point, I defer to you and Keith and I stay away from apt-get to add it.

In order to avoid anything non standard on the ATOM, is it worth us asking nicely for a batch file that can be run to give simple data reports for you to lift via your secure network? Or have you already got this?

I'd possibly suggest a batch file running from cron/at for two days midweek, and another over the weekend, that would store the two numbers (RX/TX) in a file. Or alternatively, a weekly/monthly report to send to the station maintainer. The first avoids a processing overhead all the time.

If you are going to terminal, then you need to learn (for those who haven't) to pipe the output of a command to more or grep

Code: [Select]
pi@ognpaw:~ $ dpkg -l | more
or

Code: [Select]
dpkg -l | grep tcp
more means you only get one screen full until you press the space bar for the next screen.

grep takes the output of your command and filters it against the string specified after grep.



Title: Re: Data consumption
Post by: Admin on March 02, 2021, 03:59:02 pm
There is a dedicated network for the traffic that is sent over the GRID
this is on the interface ztmfcr5xhu - this will indicate exactly what has been sent to/from the grid

Of course there is also traffic being sent to the OGN servers, which is sent over the WAN interface which we do not control

looking at my own interfaces

Code: [Select]
eth0      Link encap:Ethernet  HWaddr
          RX packets:1399083 errors:0 dropped:337 overruns:0 frame:0
          TX packets:1665028 errors:0 dropped:0 overruns:0 carrier:0
          RX bytes:209033288 (199.3 MiB)  TX bytes:623734832 (594.8 MiB)

ztmfcr5xhu Link encap:Ethernet  HWaddr 0e:5c:be:62:ea:54
          RX packets:202265 errors:0 dropped:0 overruns:0 frame:0
          TX packets:622153 errors:0 dropped:0 overruns:0 carrier:0
          RX bytes:28895487 (27.5 MiB)  TX bytes:61981272 (59.1 MiB)

In my environment there is about 10X more traffic on the local network
don't forget that the interface will report broadcasts and multicasts, which may / may not be destined for this interface.

Thx
Lee
Title: Re: Data consumption
Post by: steveu on March 11, 2021, 02:23:53 pm
Just to keep this thread alive with the distinction Lee made for us:

Code: [Select]
eth0     
          Link encap:Ethernet  HWaddr b8:27:eb:76:db:9c                                                                                                                                                                                                                                                     
          RX packets:2379890 errors:0 dropped:1172 overruns:0 frame:0                                                                                                                     
          TX packets:4620696 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                     
          collisions:0 txqueuelen:1000                                                                                                                                                   
          RX bytes:660552702 (629.9 MiB)  TX bytes:1232397832 (1.1 GiB)

Code: [Select]
ztmfcr5xhu
          Link encap:Ethernet  HWaddr 0e:72:51:23:62:c6                                                                                                                                                                                                                                                     
          RX packets:976389 errors:0 dropped:0 overruns:0 frame:0                                                                                                                         
          TX packets:241374 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                       
          collisions:0 txqueuelen:1000                                                                                                                                                   
          RX bytes:146525442 (139.7 MiB)  TX bytes:55246748 (52.6 MiB)

We're sending a lot to the OGN.

As this is a 6 day period, I would multiply by 5 for 30 days, and get about 5.5GB.

Were I on a mobile broadband tariff, I'd go for about 10GB, and keep an eye on it as traffic density is still very low.