#364822 - 20/09/2015 19:19
Linux PC stalls for half-second frequently
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
I have a Linux desktop PC, and recently it has started "stalling" for (what feels like) ~200-500ms. This occurs every few minutes. If I'm (e.g.) watching a Youtube video, the video stalls, but the audio doesn't. It'll happen for other operations: it just happened while I was typing this post, in fact.
This is irritating. It also occasionally happens while playing Kerbal Space Program, and that could be fatal to my Kerbals.
Plus: it only just started happening (last couple of days), which makes me concerned that something expensive is on its way out.
Any ideas for Linux-y places I could look to see what's happening?
_________________________
-- roger
|
Top
|
|
|
|
#364823 - 21/09/2015 00:20
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
First place I'd start is the system log, see if anything of note pops into the log right before, or right after these stalls happen.
Depending on your distro, it could be /var/log/syslog or /var/log/messages
Keep a terminal window up and a tail -f on the log.
|
Top
|
|
|
|
#364824 - 21/09/2015 01:33
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
distros tend to be dumb about logs --> they file them into all kinds of strange individual files.
So just look at the kernel logs, the most recent of which will still be in the RAM FIFO. Use 'dmesg' to dump them out and look for trouble there.
|
Top
|
|
|
|
#364826 - 21/09/2015 09:42
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
new poster
Registered: 03/04/2005
Posts: 48
Loc: Fairbanks, Alaska
|
One of my favorite tools (back in the day) was to do
top
and see if there was a process that was devouring CPU or RAM.
_________________________
Tom C
|
Top
|
|
|
|
#364827 - 21/09/2015 10:06
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
pooh-bah
Registered: 12/01/2002
Posts: 2009
Loc: Brisbane, Australia
|
Sounds like a drive dying to me but logs mentioned will tell you this pretty quickly.
_________________________
Christian #40104192 120Gb (no longer in my E36 M3, won't fit the E46 M3)
|
Top
|
|
|
|
#364828 - 21/09/2015 12:50
Re: Linux PC stalls for half-second frequently
[Re: mlord]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Use 'dmesg' to dump them out and look for trouble there. Seeing a lot of this reported bug, but the times don't correlate. I'm going to upgrade to a newer kernel anyway...
_________________________
-- roger
|
Top
|
|
|
|
#364829 - 21/09/2015 12:52
Re: Linux PC stalls for half-second frequently
[Re: Shonky]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Sounds like a drive dying to me but logs mentioned will tell you this pretty quickly. I'm not seeing anything in the logs. I have a 120GB OCZ Agility 3 as my main drive, and a pair of 1TB Western Digital Caviar Green disks as my data drive. I've got an encryptfs private directory, but whether I'm using it or not doesn't seem to correlate.
_________________________
-- roger
|
Top
|
|
|
|
#364830 - 21/09/2015 12:54
Re: Linux PC stalls for half-second frequently
[Re: aksnowbiker]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
One of my favorite tools (back in the day) was to do
top
and see if there was a process that was devouring CPU or RAM.
It's a Core i7 with 12 logical cores. That'd have to be using a *lot* of CPU. Load average is 0.36, 0.48, 0.49, btw. I've got 15GB free memory (of which 11GB is cache).
_________________________
-- roger
|
Top
|
|
|
|
#364834 - 21/09/2015 17:52
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
new poster
Registered: 03/04/2005
Posts: 48
Loc: Fairbanks, Alaska
|
I wonder if you would see the same halting behavior if you stopped X and operated the machine solely from the shell session. You'd have to write a quick script that did some quick calculation ad infiniteum (add 1 to the previous sum, for example) and let it run for a few minutes, long enough to see if it did the halt thing.
If it works fine without halting, that would make me think that the trouble has to do with a package somewhere, and not the kernel. Then again, I go barking up the wrong tree with alarming regularity.
_________________________
Tom C
|
Top
|
|
|
|
#364836 - 22/09/2015 01:48
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
I have a 120GB OCZ Agility 3 as my main drive, and a pair of 1TB Western Digital Caviar Green disks as my data drive. Ah. The WD Green drives default to a rather aggressive power-saving strategy that conflicts badly with Linux access patterns. You _could_ be seeing jerky behaviour as a result of Linux having to wait for the drive(s) to wake up again. There exists a MS-DOS (seriously!) utility from WD that can modify the "WDIDLE3" timeout to a more suitable value (default is 8 seconds, much better is something like 30 seconds). If you're not into DOS, and not as fussy about warranty, then my own hdparm utility for Linux has the -J flag to do a very similar thing. Eg. "hdparm -J30 /dev/sdX". The settings change need only be done once, and is permanent unless modified again.
|
Top
|
|
|
|
#364837 - 22/09/2015 01:52
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
Another possibility is if the OCZ Agility drive has been mounted with the "discard" flag set (look in /proc/mounts for "discard").
Despite my best advice at the time, other kernel devs moved to make "discard" (aka. "on the fly TRIM") the default setting in some distros, even though many/most drives of the time could be rather slow at doing TRIM, including the Agility-3 series.
My main notebook/system here runs entirely from an Agility-3 drive, and never stalls or stutters. But I ensure it never gets mounted with a "discard" flag set, preferring instead to run my own "wiper.sh" script once every few weeks to catch up on TRIM operations. The newer "fstrim" utility does the same sort of thing these days.
Edited by mlord (22/09/2015 01:57)
|
Top
|
|
|
|
#364838 - 22/09/2015 07:33
Re: Linux PC stalls for half-second frequently
[Re: mlord]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Another possibility is if the OCZ Agility drive has been mounted with the "discard" flag set (look in /proc/mounts for "discard"). Not set. I'm using Linux Mint 17, if that makes a difference. Eg. "hdparm -J30 /dev/sdX". The settings change need only be done once, and is permanent unless modified again. Done. I'm not convinced that this was the problem: my "working set" of data is on the SSD -- the WD drives are only used for big stuff: downloads, games, movies, music, etc. Also, it just hiccuped again while typing this: immediately after I made the change. Oh, and because this behaviour only just started happening. I've had the PC for about 2 years now. I haven't rebooted into the new kernel yet, though. Some more information: the PC itself doesn't seem to hang. That is: no process seems to be affected. Keystrokes aren't dropped, as far as I can tell; they just seem to be delayed. Well, either they're delayed, or the video output corresponding to the keystrokes is delayed. Oh, the graphics card:
$ lspci | grep VGA
03:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 760] (rev a1)
$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 331.113 Mon Dec 1 21:08:13 PST 2014
GCC version: gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
_________________________
-- roger
|
Top
|
|
|
|
#364840 - 22/09/2015 11:39
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12338
Loc: Sterling, VA
|
...watching a Youtube video, the video stalls, but the audio doesn't. Well, either they're delayed, or the video output corresponding to the keystrokes is delayed. I would think it's the video card. When typing through one of these hiccups, when the hiccup is over does the text appear all at once, or look like it's being typed really fast? Do you have another card you could swap out and test?
_________________________
Matt
|
Top
|
|
|
|
#364841 - 22/09/2015 12:17
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
Mmmm. Well so much for the Linuxy things that it could have been. A newer kernel may have fixes for SATA Link Power Management (LPM), which *used* to cause problems like this a few years ago, but Linux Mint 17 (which I also run here, also on NVIDIA) doesn't seem to have that issue. Dunno.
|
Top
|
|
|
|
#364842 - 22/09/2015 12:19
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
I suppose, just for fun, you could run my "wiper.sh" script to tidy up the SSD internals and see if that makes a difference.
Eg. ./wiper.sh --commit /
Attachments
wiper.sh (166 downloads)Description: SSD TRIM script, from hdparm package.
Edited by mlord (22/09/2015 12:20)
|
Top
|
|
|
|
#364858 - 23/09/2015 14:50
Re: Linux PC stalls for half-second frequently
[Re: Dignan]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
I would think it's the video card. When typing through one of these hiccups, when the hiccup is over does the text appear all at once, or look like it's being typed really fast? Based on paging through a log file on another host (i.e. lots of latency), it appears that the keystrokes are instant, and the rendering is the part that's stalling. Do you have another card you could swap out and test? Not easily, no.
_________________________
-- roger
|
Top
|
|
|
|
#364862 - 23/09/2015 20:20
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
I haven't rebooted into the new kernel yet, though. Well, that was kinda scary. I rebooted, got to the Linux Mint login screen and then my keyboard and mouse didn't work. Not even Ctrl+Alt+ combos to switch VC or to kill X. ssh in from my Mac, remove the latest kernel, reboot. Sorted. I finally rebooted to try to fix texture corruption in Kerbal Space Program. I wonder if it works again...?
_________________________
-- roger
|
Top
|
|
|
|
#364864 - 24/09/2015 04:51
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
new poster
Registered: 03/04/2005
Posts: 48
Loc: Fairbanks, Alaska
|
It's amazing how often things snowball and dissolve into digital hell.
Did you determine whether the stutter happens at a VC?
_________________________
Tom C
|
Top
|
|
|
|
#364965 - 06/10/2015 16:56
Re: Linux PC stalls for half-second frequently
[Re: Dignan]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Do you have another card you could swap out and test? I do now. GTX 960 turned up this afternoon. It's been in the PC for about an hour, and seems to have fixed the problem. Of course, the challenge now is to figure out what's wrong with the old card, because -- other than that -- it works fine, and it's quite a good card.
_________________________
-- roger
|
Top
|
|
|
|
#364988 - 07/10/2015 13:50
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
I suppose it could have something to do with the NVIDIA card/driver switching between performance/thermal levels on the fly. A newer/faster card might never need to switch.
|
Top
|
|
|
|
#364989 - 07/10/2015 15:46
Re: Linux PC stalls for half-second frequently
[Re: mlord]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
I suppose it could have something to do with the NVIDIA card/driver switching between performance/thermal levels on the fly. A newer/faster card might never need to switch. Yeah. I considered that: I had the nvidia settings GUI open while I was investigating this on the old card; there weren't obvious thermal spikes or anything, and it didn't _seem_ correlated with the performance boost stuff. I say _seem_, because the sampling interval on the performance display in the GUI is so long as to be worthless for this kind of thing.
_________________________
-- roger
|
Top
|
|
|
|
#364990 - 07/10/2015 17:21
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
new poster
Registered: 03/04/2005
Posts: 48
Loc: Fairbanks, Alaska
|
If it was switching between performance levels based on thermal conditions, wouldn't the cool-down period be longer than what has been described?
_________________________
Tom C
|
Top
|
|
|
|
#365000 - 08/10/2015 18:08
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
I do now. GTX 960 turned up this afternoon. It's been in the PC for about an hour, and seems to have fixed the problem. I spoke too soon; same problem. Maybe it's correlated to how long the PC's been up, or maybe to load or something. One thing occurs: I have VirtualBox on here; maybe that's doing odd things...
_________________________
-- roger
|
Top
|
|
|
|
#365058 - 14/10/2015 16:49
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
new poster
Registered: 03/04/2005
Posts: 48
Loc: Fairbanks, Alaska
|
But the "halt" occurs even if VirtualBox isn't running, right? I assume that you can run your host OS without running VB. (I've never played with it before -- it looks intriguing.)
I still can't help but wonder if the problem would still occur if X was killed and you tested for the problem at the console. Just using one of the VC's would leave X running and not really tell you anything. If the problem disappears along with X, it might be worth upgrading/downgrading/renewing X.
_________________________
Tom C
|
Top
|
|
|
|
#365059 - 14/10/2015 16:54
Re: Linux PC stalls for half-second frequently
[Re: aksnowbiker]
|
carpal tunnel
Registered: 20/12/1999
Posts: 31597
Loc: Seattle, WA
|
But the "halt" occurs even if VirtualBox isn't running, right? I had VirtualBox mess up my system in an interesting way even when it wasn't running. The feature of Windows 8 which allows you to connect to a wireless projector via WiDi/Miracast was completely broken by having VirtualBox installed at all. I know Linux isn't Windows. I'm just saying it's possible that VirtualBox can do unexpected things to a system even when it's not actively running.
|
Top
|
|
|
|
#365060 - 14/10/2015 17:22
Re: Linux PC stalls for half-second frequently
[Re: tfabris]
|
carpal tunnel
Registered: 13/02/2002
Posts: 3212
Loc: Portland, OR
|
I know Linux isn't Windows. I'm just saying it's possible that VirtualBox can do unexpected things to a system even when it's not actively running. VirtualBox's kernel modules on Linux have historically been bad enough that starting in 2011, they've been flagged as tainting the kernel. I don't know if the code has improved since then. But it wouldn't hurt to verify if the stalls are happening even when VirtualBox's kernel modules aren't loaded.
|
Top
|
|
|
|
#365066 - 15/10/2015 08:16
Re: Linux PC stalls for half-second frequently
[Re: aksnowbiker]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
I still can't help but wonder if the problem would still occur if X was killed and you tested for the problem at the console. That doesn't really help, because I need to use the computer. I guess I could try running tmux on a VC...
_________________________
-- roger
|
Top
|
|
|
|
#365067 - 15/10/2015 08:49
Re: Linux PC stalls for half-second frequently
[Re: aksnowbiker]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
I assume that you can run your host OS without running VB I'll need to move my website, which would be annoying, but is probably something I should do anyway.
_________________________
-- roger
|
Top
|
|
|
|
#365068 - 15/10/2015 13:55
Re: Linux PC stalls for half-second frequently
[Re: Roger]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
How do you have VirtualBox configured in regards to VT-d or other virtualization hardware support? (VT-x, etc)
May be worth flipping them to the opposite of now. VT-d would let the VM also have direct PCI access to the GPU, and could be causing a bit of a hitch.
|
Top
|
|
|
|
|
|