YAMon 3.0... Usage Reporting by device per hr/day/mo

Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> General Questions
Goto page Previous  1, 2, 3 ... 31, 32, 33 ... 149, 150, 151  Next
Author Message
David Lewis
DD-WRT Novice


Joined: 12 Mar 2016
Posts: 6

PostPosted: Thu Jun 23, 2016 19:00    Post subject: Reply with quote
al_c wrote:
MichaelH99 wrote:
I'm just guessing but maybe the script doesn't start back up if the shutdown script didn't get run?
next time it happens, can you check if /tmp/yamon3-running/(?) exists... It should not (because it's in tmp) but if it does, that'll stop it from starting properly


Luckily (!) a lightning strike just took out the electricity supply for a minute, providing an opportunity to investigate.

I can report that YAMon3 wasn't running after power was restored, and there was no /tmp/YAMon* file or folder present. Starting the script using startup.sh worked fine, after which there was a /tmp/YAMon3-running folder present.

What should I look for next time?
Sponsor
gotovamsee
DD-WRT Novice


Joined: 07 May 2016
Posts: 13

PostPosted: Thu Jun 23, 2016 19:53    Post subject: Reply with quote
Gosh! As I worried, all the YAMON3 data is wiped from /OPT with an unexpected reboot of the router.

I tried binding it on mnt folder instead of /tmp folder this time using: mount -o bind /mnt/sda1/opt /opt

Worked but not holding the setup after a reboot. I have to copy the YAMON3 bakcup folder into /OPT and run restart.sh.


Gurus please help, this is killing all my time now.
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jun 23, 2016 20:52    Post subject: Reply with quote
David Lewis wrote:
al_c wrote:
MichaelH99 wrote:
I'm just guessing but maybe the script doesn't start back up if the shutdown script didn't get run?
next time it happens, can you check if /tmp/yamon3-running/(?) exists... It should not (because it's in tmp) but if it does, that'll stop it from starting properly


Luckily (!) a lightning strike just took out the electricity supply for a minute, providing an opportunity to investigate.

I can report that YAMon3 wasn't running after power was restored, and there was no /tmp/YAMon* file or folder present. Starting the script using startup.sh worked fine, after which there was a /tmp/YAMon3-running folder present.

What should I look for next time?

What do you have in the startup command in the DD-WRT GUI --> Administration --> Commands?

Al
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Jun 23, 2016 20:59    Post subject: Reply with quote
gotovamsee wrote:
Gosh! As I worried, all the YAMON3 data is wiped from /OPT with an unexpected reboot of the router.

I tried binding it on mnt folder instead of /tmp folder this time using: mount -o bind /mnt/sda1/opt /opt

Worked but not holding the setup after a reboot. I have to copy the YAMON3 bakcup folder into /OPT and run restart.sh.


Gurus please help, this is killing all my time now.

You cannot install YAMon in /tmp/ because /tmp is volatile memory that is erased every time the router reboots.

You need to have permanent location for YAMon and it's data... the easiest solution is a USB drive connected to your router. I recommend that it is formatted to ext4 (although some routers may only support ext3). Name the USB volume opt and it should auto-mount to that name. Otherwise, make sure that you have the proper settings in the DD-WRT Admin GUI --> Services(?) --> USB.

If you have set things up in this way and the contents of /opt do not survive a reboot, you have a completely different problem (completely unrelated to YAMon).

However, I'm hardly a guru...

Al
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Fri Jun 24, 2016 2:38    Post subject: Re: Power cut Reply with quote
cnone wrote:
After every power cut, I have to start script manually. Why is this happening Sad
If I intentionally reboot the router everything works as expected.

Latest script on TP-Link WR1043ND V2

I noticed that this was happening on my router too...

I don't if this'll work for you as well but I deleted and resaved the startup script and things seem to work properly again - i.e., after I unplug the router, YAMon starts as expected... can you try this too?

Steps:
open the DD-WRT GUI and change to the Administration-->Commands tab
click `Edit` in the startup fieldset
cut the text (mine was `/opt/YAMon3/startup.sh 10`) so that the Commands field is empty
click `Save Startup`

paste the text back into the Commands field and click `Save Startup` again

Either reboot your router or wait for a power failure (or nearby lightening strike)... does YAMon start properly now?

(It does for me?!?)

Al
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1464

PostPosted: Fri Jun 24, 2016 7:36    Post subject: Reply with quote
Hey Al, what keeps wireless clients from showing up in YAMon on my R8500?

Dave

_________________
Segment 1 XR700 10Gb LAN, 1Gb WAN ISP BS
Wired AP 1 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 2 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 3 Unifi Wifi 6 LR US 1Gb LAN
Syslog Services Asustor 7110T NAS 10GB
NetGear XS716T 10GB Switch
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
YAMon https://usage-monitoring.com/index.php
MichaelH99
DD-WRT Novice


Joined: 21 Aug 2010
Posts: 8

PostPosted: Fri Jun 24, 2016 16:21    Post subject: Reply with quote
I hate posting this because the answer is probably somewhere in the 32 pages of this thread but that's too much to wade through (oh, for thread search!)

Is there older documentation that I can use for figuring out how best to use yamon? The 3.0 site has just "coming soon" for anything that's not for installation.

The two biggest questions I have are:

1) How can I identify the source of regular spikes I see in cpu/packet stats? The fact that they're happening is good to know but unless I can see what the traffic was, I have no hope of figuring out if it's benign (stupid windows updates) or something I should mallet.

2) How can I figure out why I'm regularly seeing a 150% delta between yamon and router stats?

Thanks in advance...
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Fri Jun 24, 2016 16:46    Post subject: Reply with quote
ddaniel51 wrote:
Hey Al, what keeps wireless clients from showing up in YAMon on my R8500?

Dave


Was YAMon seeing them before? Are these new devices? Are they IPv6? & do you have IPv6 enabled in YAMon?

Sorry - no answers, just questions.

Al
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Fri Jun 24, 2016 17:35    Post subject: Reply with quote
MichaelH99 wrote:
I hate posting this because the answer is probably somewhere in the 32 pages of this thread but that's too much to wade through (oh, for thread search!)

Is there older documentation that I can use for figuring out how best to use yamon? The 3.0 site has just "coming soon" for anything that's not for installation.

The two biggest questions I have are:

1) How can I identify the source of regular spikes I see in cpu/packet stats? The fact that they're happening is good to know but unless I can see what the traffic was, I have no hope of figuring out if it's benign (stupid windows updates) or something I should mallet.

2) How can I figure out why I'm regularly seeing a 150% delta between yamon and router stats?

Thanks in advance...


Michael - when I have time, I'll fix usage-monitoring.com (but the road to hell is paved with good intentions).

YAMon should be able to help in a variety of ways (hopefully)... certainly in terms of gross numbers.

The hourly chart at the bottom of the Daily Usage tab gives a good indication of which devices are using how much data and when.

The Live Usage tab allows you to see which IP addresses each device is connecting to (and you can click to do a reverse IP lookup for each of the connections).

As for the difference between YAMon's numbers and the router's, that is a huge discrepancy.

Are PrivOxy and/or UPnP enabled on your router? (they are know to interfere with the numbers)

Do you have entries in your iptables rules that hijack data before YAMon sees it?

Do you have IPv6 traffic on your router? Is YAMon properly set up for that traffic?

Does that help at all?

Again, only questions, no answers.

Al
MichaelH99
DD-WRT Novice


Joined: 21 Aug 2010
Posts: 8

PostPosted: Fri Jun 24, 2016 18:09    Post subject: Reply with quote
al_c wrote:
MichaelH99 wrote:
I hate posting this because the answer is probably somewhere in the 32 pages of this thread but that's too much to wade through (oh, for thread search!)

Is there older documentation that I can use for figuring out how best to use yamon? The 3.0 site has just "coming soon" for anything that's not for installation.

The two biggest questions I have are:

1) How can I identify the source of regular spikes I see in cpu/packet stats? The fact that they're happening is good to know but unless I can see what the traffic was, I have no hope of figuring out if it's benign (stupid windows updates) or something I should mallet.

2) How can I figure out why I'm regularly seeing a 150% delta between yamon and router stats?

Thanks in advance...


Michael - when I have time, I'll fix usage-monitoring.com (but the road to hell is paved with good intentions).

YAMon should be able to help in a variety of ways (hopefully)... certainly in terms of gross numbers.

The hourly chart at the bottom of the Daily Usage tab gives a good indication of which devices are using how much data and when.

The Live Usage tab allows you to see which IP addresses each device is connecting to (and you can click to do a reverse IP lookup for each of the connections).

As for the difference between YAMon's numbers and the router's, that is a huge discrepancy.

Are PrivOxy and/or UPnP enabled on your router? (they are know to interfere with the numbers)

Do you have entries in your iptables rules that hijack data before YAMon sees it?

Do you have IPv6 traffic on your router? Is YAMon properly set up for that traffic?

Does that help at all?

Again, only questions, no answers.

Al


No worries, I know about not having enough time for everything.

Unfortunately the live usage doesn't tell me what the machines are trying to do. Is there any raw data I can pull to tell me what a machine was doing in past?

I've got ip6 and upnp disabled.

I don't have any custom iptables rules set up. I've avoided messing with the settings at that level.

I hadn't heard of PrivOxy until you mentioned it.
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1464

PostPosted: Sat Jun 25, 2016 1:51    Post subject: Reply with quote
al_c wrote:
ddaniel51 wrote:
Hey Al, what keeps wireless clients from showing up in YAMon on my R8500?

Dave


Was YAMon seeing them before? Are these new devices? Are they IPv6? & do you have IPv6 enabled in YAMon?

Al


No IPV6. I did some more tests on the radios on the R8500 and R7000. It even appeared the 5ghz was dead while the 2.4ghz was working, intermittently. It just didn't make sense. It was displaying increasing router traffic totals without incrementing individual ones.

I have been using a users.js copied from the main router and in use since Y2. It is a convoluted monster.

I finally stopped Y3, erased the users.js and today's hourly then restarted Y3. I've started and stopped Y3 a couple of times now to edit the users.js names. It is now displaying data properly on all radios.

I'd like to see Y3 read the users.js periodically so we could make realtime edits on the fly.

Tnx
Dave

_________________
Segment 1 XR700 10Gb LAN, 1Gb WAN ISP BS
Wired AP 1 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 2 Unifi Wifi 6 LR US 1Gb LAN
Wired AP 3 Unifi Wifi 6 LR US 1Gb LAN
Syslog Services Asustor 7110T NAS 10GB
NetGear XS716T 10GB Switch
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
YAMon https://usage-monitoring.com/index.php
tsaylor
DD-WRT Novice


Joined: 20 Jan 2014
Posts: 19

PostPosted: Sun Jun 26, 2016 5:26    Post subject: Reply with quote
Greetings Al! (and also any Tomato users out there)

I recently converted to Tomato (after verifying that I could still use YAMon, since I can't live without it anymore). Upon installing I noticed that the support doesn't seem to be 100% there yet, and I've also seen a few other users asking, so I spent some time figuring out what is wrong. Hopefully this will be useful to other Tomato users, and hopefully to Al for adding full support.

Here are the issues I ran into under Tomato (using dd-wrt as the firmware choice in the settings, for now):

1) Different locations for _dnsmask_conf and _dnsmask_leases (config.file settings):
_dnsmasq_conf="/tmp/etc/dnsmasq.conf"
_dnsmasq_leases="/tmp/var/lib/misc/dnsmasq.leases"

2) Different location for www files (config.file setting):
_wwwPath="/www/ext/"

3) Issues with the getDeviceName() logic within the yamon script (which only affects the device name detection when adding new devices to users.js). I found that the "Asuswrt" logic (firmware "2") in that function seemed to do the trick as long as I changed the nvram setting to "dhcpd_static=" in this line:
local nvr=$(nvram show 2>&1 | grep -i "dhcpd_static=")
Also the _dnsmasq_conf logic in that section was only able to retrieve the machine ip in my case rather than the machine name (but perhaps that is expected or common, since that name seems to be optional)

4) A bit of struggle about where to put the startup/shutdown scripts. Initially I used the USB "before mounting" script, but found that it would fail because the time wasn't set yet on the router, and it actually couldn't be set yet since the WAN isn't up at that point. Eventually I moved my startup script (which contains YAMon3/startup.sh) to the Administration scripts for "WAN Up" which solved that issue. I left the YAMon3/shutdown.sh command in the USB "Run before unmounting" section because it seemed to me like the most logical place. For reference, those two script settings are stored in nvram as "script_wanup" and "script_usbumount".

With those changes, and as far as I can tell after very limited testing, everything is working well. Very Happy
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Sun Jun 26, 2016 14:12    Post subject: Reply with quote
tsaylor wrote:
Greetings Al! (and also any Tomato users out there)

I recently converted to Tomato (after verifying that I could still use YAMon, since I can't live without it anymore). Upon installing I noticed that the support doesn't seem to be 100% there yet, and I've also seen a few other users asking, so I spent some time figuring out what is wrong. Hopefully this will be useful to other Tomato users, and hopefully to Al for adding full support.

Here are the issues I ran into under Tomato (using dd-wrt as the firmware choice in the settings, for now):

1) Different locations for _dnsmask_conf and _dnsmask_leases (config.file settings):
_dnsmasq_conf="/tmp/etc/dnsmasq.conf"
_dnsmasq_leases="/tmp/var/lib/misc/dnsmasq.leases"

2) Different location for www files (config.file setting):
_wwwPath="/www/ext/"

3) Issues with the getDeviceName() logic within the yamon script (which only affects the device name detection when adding new devices to users.js). I found that the "Asuswrt" logic (firmware "2") in that function seemed to do the trick as long as I changed the nvram setting to "dhcpd_static=" in this line:
local nvr=$(nvram show 2>&1 | grep -i "dhcpd_static=")
Also the _dnsmasq_conf logic in that section was only able to retrieve the machine ip in my case rather than the machine name (but perhaps that is expected or common, since that name seems to be optional)

4) A bit of struggle about where to put the startup/shutdown scripts. Initially I used the USB "before mounting" script, but found that it would fail because the time wasn't set yet on the router, and it actually couldn't be set yet since the WAN isn't up at that point. Eventually I moved my startup script (which contains YAMon3/startup.sh) to the Administration scripts for "WAN Up" which solved that issue. I left the YAMon3/shutdown.sh command in the USB "Run before unmounting" section because it seemed to me like the most logical place. For reference, those two script settings are stored in nvram as "script_wanup" and "script_usbumount".

With those changes, and as far as I can tell after very limited testing, everything is working well. Very Happy


Todd - Good to hear from you again. I'm thrilled to hear that you are still using my little script.

Thank you very much the detailed explanation! I do not have the time (or budget) to test every firmware variant and/or router combination and therefore rely upon updates like yours to broaden the reach of YAMon.

I will fold your updates into the code base!

Al
tsaylor
DD-WRT Novice


Joined: 20 Jan 2014
Posts: 19

PostPosted: Sun Jun 26, 2016 16:03    Post subject: Reply with quote
al_c wrote:

Todd - Good to hear from you again. I'm thrilled to hear that you are still using my little script.

Thank you very much the detailed explanation! I do not have the time (or budget) to test every firmware variant and/or router combination and therefore rely upon updates like yours to broaden the reach of YAMon.

I will fold your updates into the code base!
Al


Thanks Al! Glad you remembered me, as it has been quite a while. I'd been running version 2 all along, but when I semi-soft-bricked my router with a small config change I decided to do several things I'd been planning. I ended up on Tomato because it seemed more stable on my particular router (R8000) and even a bit easier to use, and of course I moved to YAMon3 which is looking great! My other big move is to segment my network to keep guests and "internet of things" separate from my more important systems.

Anyway, I'm glad to test any changes you fold in, or help in any other way I can. Thanks as always for such an excellent tool!

- Todd
Howick
DD-WRT Novice


Joined: 27 Jun 2016
Posts: 2

PostPosted: Tue Jun 28, 2016 14:52    Post subject: Reply with quote
Al

Thank you so much for an extremely helpful tool Very Happy

After reading the last 32 pages I can't help but feel that you have made a lot of work for yourself to track down and fix errors - many of which are user errors Wink

I have to say that after two false starts (one mostly successful onto a ntfs formatted usb drive, and then usb as recommended but still no good because the dd-wrt firmware needed updating) the installation went perfectly.

I have noticed that I can't seem to save my details about the "Bonus Data allowance cap". Whenever I refresh the browser page and then open up the settings tab neither radio button is selected and the "50" I have typed in for the GB has disappeared. This happens on Linux, Android and Windoze. Does it make any difference to the presentation of the data?

I would also be interested in finding out where the ISP data is stored and if it can be viewed/adjusted without re-importing the whole month.

When it comes to the format for importing, would it be feasible to set up a conversion script that can be user specific? I can see that if everyone were to get you to add another format for each ISP, then the list of formats would be huge. For some, such as mine, it would just be a change in the presentation of the date. At present I just use excel to alter the date format. I am thinking of something that would set up a script when I first try to process data. Any future occassions would automatically convert the date format whenever I press the "process" button.

Once again many thanks
Bruce

P.s. One other thing that is bugging me. The reports all have headings alluding to "unlimited" data. As much as I would love to have unlimited off-peak, my satellite internet providers won't do it - at any price! I realize that it is only semantics but could it be reworded to "Off-Peak" or similar?



Capture.PNG
 Description:
 Filesize:  8.97 KB
 Viewed:  8449 Time(s)

Capture.PNG


Goto page Previous  1, 2, 3 ... 31, 32, 33 ... 149, 150, 151  Next Display posts from previous:    Page 32 of 151
Post new topic   This topic is locked: you cannot edit posts or make replies.    DD-WRT Forum Index -> General Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum