Bad performance under heavy I/O, and a possible solution

[Shador]

Do you mean fair as in FCFSFIFO, because that's the definition of fair? I think this would give you poor performance --> high response time and probably would also create problems with audio playback. The problem is that schedulers are always a compromise. For example CFQ has apparently good response times, which is generally very important for schedulers of any kind. Another scheduler you haven't mentioned yet is BFQ, which has worse response times than CFQ but a higher throughput, which can be less of a problem on desktop systems. You could try using BFQ, I have it on one system here, but I never had problems with IO really anyway.

I don't think this is much of a problem of Linux anyway. HDDs are just the slowest part of the computer and this is one other aspect of the von Neumann bottleneck. It would be interesting to know how much RAM you have on the affected systems. I tend to use much as it significantly improves interactivity.

[GJones]

Apologies for the comment about Linux being crappy (it isn't, it's just full of bugs like every other OS).

I think you're probably right, as some messing around with Windows shows that it suffers from this at least as much as Linux - probably I didn't notice because I don't do usually do any serious high I/O stuff on Windows. Oh well.

[witek]

Regarding bad i/o performance, yes, Linux is getting more and more crappy. I`m using Fedora at this moment and I`m trying to copy one big file to my USB pendrive. During this my desktop locks up from time to time. I used to use 'ionice' to schedule i/o consuming operations but recent Linux distros like Fedora and Ubuntu seems to ignore it, same as good old 'nice' command. Salix seems to respect these yet so you might try:

Code: Select all

ionice -c3 nice -n 19 your_command

in order to make it least i/o consuming.

[GJones]

Hmm... Now if only the system would do that to IO hogs automatically.

And I'm not sure "crappy" is the word, but something bad is definitely happening. When I was in high school I ran Arch Linux with KDE3 on an IBM 300 PL, and performed better than Windows 98 on the same machine. And today I've got Debian Squeeze on a Mac G3, and playing an MP3 brings it to its knees. I like some of the new features in recent Linux kernels; but it looks like they're bought at the price of worse performance and more bugs.

(One can of course compile one's own kernel, but that takes a long time even on current machines, and is a PITA too. And I just don't think it should be necessary to customize a kernel by hand just to get decent performance on a machine that was perfectly usable 5 years ago.)

[GJones]

I think I've found another solution... Might be a little dangerous on desktops though.

Code: Select all

vm.dirty_writeback_centisecs = 1500

(The default for this value appears to be 500 for most kernels.)

The way I understand it, the kernel by default forces data in memory to be written to the disk every 5 seconds. This changes that interval to 15 seconds. Presumably increasing the value results in shorter bursts of disk I/O. Anyway the difference in performance is huge; with increased dirty_writeback_centisecs, programs are as responsive during heavy I/O as when nothing is happening at all.

However, less frequent writes to the disk mean bigger potential for data loss. If you are not using a laptop, and do not have a UPS, you might want to avoid this tweak. Also, the ext3 filesystem IIRC has its own method of forcing commits every 5 seconds, and I'm not sure how that would interact with this change.