All Questions
Tagged with performance io
99
questions
1
vote
1
answer
55
views
Disk saturated by mariadb on CentOS 7, blocks everything
For the past few days I have been having a problem with a webserver running CentOS 7. Every 13-14 minutes some operation (which I identified with mariadb using iostat) seems to completely saturate the ...
1
vote
0
answers
66
views
Couchdb 3.1.1 increases disk latency gradually over weeks
I've been dealing with this issue for weeks now.
I have the followin scenario:
couchdb2.3.1-A <===> couchdb2.3.1-B <===> couchdb3.1.1-A <===> couchdb3.1.1-B
where <===> ...
-1
votes
1
answer
427
views
Simulate some process IO with fio or other tool
Is there any way to track a specific process and get stats for disk IO such as queue depth, total reading/writing threads, percent of reads/writes and so on?
The main goal is to use all that ...
- 103
2
votes
1
answer
404
views
About IOPS of a RAID5 disk array without cache
we are using a 530-8i RAID adapter,which is none-cache supported, on a Lenovo ThinkSystem SR590 server.And we have 4 HDD(600GB/10K RPM/SAS) in the array.
I read an article about calculating the disk/...
- 21
2
votes
0
answers
612
views
Access disk io data in linux without tools like iotop/iostat to find processes or files with IO
We have regularly high load average. With the help of a zabbix agent, I could narrow it down to be mainly because of cpu iowait (and lastly disk io). I am not allowed to install any additional ...
- 121
0
votes
1
answer
277
views
Is RAID 6 a good choice regarding performance on server used for computation?
We are operating a server for doing computations with large data sets (100GB - 5TB) for a small group of researchers (10-20 people). At the moment we are running 4x8TB spinning drives in a RAID 6 ...
- 234
2
votes
1
answer
104
views
MYSQL vs. InnoDB [closed]
mysqltuner output.
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MEMORY +MRG_MYISAM +...
1
vote
0
answers
140
views
disk I/O latency spikes (contains log files from amplify)
Ubuntu 18.04 LTS server, disk is ssd. Application stack is: mongodb, node, nignx and some python.
Server is up 1 month and already had 3 OS disk fails where Ubuntu would go in read only mode. After ...
1
vote
0
answers
674
views
How does avgqu-sz size in iostat output relate to r_await and w_await?
Can someone explain what does avgqu-sz mean in the output of iostat? The manual page states like this.
avgqu-sz:
The average queue length of the requests that were issued to the device.
Can ...
- 333
1
vote
2
answers
6k
views
RAID 50 vs RAID 10 for performance?
I have a server with 6 SSD's and a raid controller card that supports both RAID 10 and RAID 50 the plan is to use this as our Build server. It will be pulling in NPM Packages and many small code files,...
- 13
0
votes
1
answer
155
views
jdb2 spike every hour for few seconds
I get IO hang up every hour on the exact same time on my Ubuntu 18.04 server.
iotop indicate that jbd2 spike to araong 30% around this time.
The hangup last only few seconds.
I started noticing this ...
1
vote
1
answer
215
views
Bandwidth and IO issues when running hardware raid and zfs
I've inherited a setup using Linux Ubuntu 14, a megaraid_sas, and ZFS.
We're getting some performance problems (never being able to fully utilize the 6GB bandwidth from the raid) and I'm curious if ...
- 535
8
votes
2
answers
19k
views
what is the relation between block size and IO?
I have been reading about disk recently which led me to 3 different doubts. And I am not able to link them together. Three different terms I am confused with are block size, IO and Performance.
I ...
- 233
1
vote
0
answers
451
views
Give read operations priority over write
I'm working on a system where I have a program that outputs high quality videos for a tv connected to an HDMI (raw 1080p data for instance) and I'm having an issue that when I copy some files to my ...
- 41
3
votes
1
answer
988
views
Linux: how can I determine if an application is doing sequential or random disk I/O?
Everyone knows that databases tend to do lots of small random I/O while big data things like Kafka tend to do large sequential I/O, but if I'm approaching this as a sysadmin without making assumptions,...
- 2,655
2
votes
1
answer
1k
views
contradictory results dd and iostat, ext4 and xfs, on EBS volumes
I'm seeing what seems to me to be contradictory results when looking at disk performance with dd versus iostat on two hosts (EC2 instances with an EBS drive). The hosts are identical except that one ...
- 2,655
5
votes
1
answer
790
views
Misunderstanding about Page Cache and dirty_background_bytes
I've been looking at this for a while now and things aren't lining up with my expectations, but I don't know if it's because something is off, or if my expectations are wrong.
So, I've got a system ...
- 161
6
votes
1
answer
917
views
RAID6 resync with fast writes but slow reads
I'm using Debian Jessie.
# uname -a
Linux host 4.9.0-0.bpo.3-amd64 #1 SMP Debian 4.9.30-2+deb9u5~bpo8+1 (2017-09-28) x86_64 GNU/Linux
And have setup a RAID6.
# cat /proc/mdstat
Personalities : [...
- 304
1
vote
1
answer
3k
views
Google compute: Why persistent ssd disks are slower than persistent standard disks?
I ran several IO benchmarks and I cannot understand how pd-ssd offers a worse IO throughtput than pd-standard.
In my tests I boot a VM with a bootable disk pd-ssd of 30 or 250GB, and I still get ...
google-cloud-platform
3
votes
1
answer
4k
views
What does the nomerge mean in Linux system?
We can see some parameters by issuing "iostat -x -d", the rrqm/s and wrqm/s stand for the merge for I/O request per second. And there is a kernel parameter controlling the I/O merge:
/sys/class/block/...
- 170
3
votes
0
answers
54
views
Do I need to create multiple filesystems on a virtual server to increase I/O performance?
I want to install Oracle on a virtual server. I don't really know the technology behind, my company just gave me a virtual server with multiple (virtual) disks.
I have the possibility to divide those ...
- 83
9
votes
2
answers
8k
views
Why do sequential writes have better performance than random writes on SSDs?
An LBA (logical block addresses) is a mapping table implemented in the FTL to match between logical and physical pages/blocks in SSDs, my guess is that most SSDs (at least when they are empty) keeps ...
- 273
10
votes
2
answers
10k
views
MySQL extremely slow on very simple SELECT queries
We have a simple web application running on a virtual machine that saves its data in a MySQL 5.5 database with the InnoDB engine. Everything worked fine for around three years, but suddenly it became ...
- 131
5
votes
2
answers
2k
views
Tuning sequential disk reads for performance
I am reading a large file sequentially from the disk and trying to understand the iostat output while the reading is taking place.
Size of the file : 10 GB
Read Buffer : 4 KB
Read ahead (/sys/block/...
- 283
0
votes
2
answers
2k
views
Poor Write I/O Performance on HP ML110 G9 Server With ESXi 6.0U2
hi everyone i need a little bit of help since i cant find a solution,
i have a HP ML110 G9 Server with:
8gb Ram
2 HDD of 2tb Each one and one SSD of 250 GB
i have installed ESXi 6.0u2 HP Customized....
- 17
1
vote
1
answer
1k
views
I/O scheduling on LVM on dm-crypt
I have the following setup:
Stock Debian stable (Linux kernel 3.16.7) running on Dell PowerEdge R320
Two SATA hard drives in RAID1 on a PERC H310 Mini controller, visible as /dev/sda
dm-crypt target ...
- 193
6
votes
3
answers
5k
views
Massive performance degradation on sustained sequential write
I'm in the process of migrating data into LUKS partitions. Now that the operating system drive is running off LUKS, I tried to start migrating the data drives. Then the server stopped responding.
...
- 314
2
votes
1
answer
276
views
Improving I/O rates for EBS snapshot backed volumes
I'm working with a system that takes a set of 42 rotating daily EBS snapshots of each of its numerous (40) volumes for disaster recovery purposes. EBS volumes are aggregated into a RAID volume. A set ...
- 11.1k
2
votes
1
answer
420
views
How can I determine which files have high I/O in Windows?
In trying to reduce the storage bottleneck on a Windows server (2008 R2) I need to find the files that have the highest I/O over a long period of time (aka 'hot' files) so I can move those to SSD ...
- 398
3
votes
2
answers
2k
views
Disk IO Rate: What causes it and what does it mean to me?
I just got a mail from Linode saying that my account "has exceeded the notification threshold (1000) for disk io rate by averaging 1017.36 for the last 2 hours. "
My question is just out of curious:
...
- 199
4
votes
1
answer
6k
views
Filesystem Performance for LUKS Encrypted Volumes?
As you can format a LUKS volume with any filesystem you want, are certain filesystems better-suited to IO performance against an encrypted filesystem? Specifically, I'm wondering about a loop device ...
- 1,157
4
votes
1
answer
2k
views
KVM bad IO sync performance in Linux guest
I'm setting up virtualized Ubuntu 15.10 guest inside Ubuntu 15.04 server using libvirt and KVM.
Storage in host system:
# mount | grep ext4
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro,...
- 141
0
votes
0
answers
620
views
How to find the bottleneck for slow rsync
I want to find the bottleneck for a slow rsync.
I see these parts (on byte flow order):
IO on source
CPU on source
Network
CPU on destination
IO on destination
How can I debug this?
- 3,603
1
vote
0
answers
39
views
Long Response Times for Reads via FCoE
We're getting some intriguing performance with our new FCoE environment, and I was hoping that people could let me know if what we're seeing is expected behaviour.
Our switching environment consists ...
- 842
1
vote
1
answer
114
views
How to check what is causing performance issue
[Edit] This question is different than what is suggested by the moderator, I am trying to find out what could have caused the API to become slow because of the few lines of codes. I am not trying to ...
- 945
1
vote
0
answers
269
views
Redis parent process smashing disk despite appendfsync = no
We are seeing loads of IO from redis-server despite our save operations being set to occur once every 15 minutes only.
The workload is a "full page cache"; we store blobs of compressed HTML. The HTML ...
- 457
0
votes
1
answer
44
views
Is "'Light-Speed' Disk Access" really a thing? [closed]
I just saw an advert for Hyve hosting on You Tube. One statement made me curious, they said they offer "'Light-Speed' Disk Access":
Is this just a fancy way of saying they have optical technology, ...
- 107
0
votes
3
answers
2k
views
I/O and RAM limitations are important for Hadoop performance. But is disk speed related to I/O?
Hortonworks says this: "Most often performance of a Hadoop cluster will not be constrained by disk speed – I/O and RAM limitations will be more important." *
How is disk speed not related to I/O ...
- 158
1
vote
0
answers
160
views
File IO being slow
I've rented this root server with a RAID1 mdadm software raid setup.
Using dd to benchmark the file IO speed it achieves 230MB/s write (default blocksize). Certain tasks such as mysql are way slower, ...
- 111
0
votes
1
answer
351
views
6 GB/s SAS RAID 10 (6 Drive) vs. 3 GB/s SAS RAID 10 (14 Drive)
I am running MS SQL 2014 Standard and have a Dell MD 3000i iSCSI SAN. Rough specs:
Dell R610 With PERC H700 controlling 6 x 2.5" 15K SAS drives (6 GB/s)
Dell MD3000i with 14 x 3.5" 15K SAS drives (...
- 409
7
votes
1
answer
2k
views
How efficient is the tac command on large files
The taccommand (catreversed) can be used to read a file backwards, just like cat reads it rom the beginning. I wonder, how efficient this is. Does it have to read the whole file from the beginning and ...
- 314
3
votes
6
answers
20k
views
Show IO on Netapp
I think I might be hitting the IO limits of what my Netapp can deliver, as I have been adding more servers to my cluster and iowait has gone up on each server.
However, how do I quantify this? How ...
- 2,273
1
vote
2
answers
609
views
What is the difference between truss -D and truss -E?
I am currently analyzing a command on Solaris that reads from a file and has very low performance. The truss -D commands shows me read system calls taking up to 0.03 seconds, but when I use truss -E, ...
- 165
2
votes
0
answers
192
views
"Nosy Neighbor" and the CFQ scheduler
I'm trying to understand how does using CFQ affects the "Nosy Neighbor" problem.
I'll ask it a bit naively, can a "Nosy Neighbor" even exist when using CFQ? Isn't that CFQ should make sure all ...
- 123
0
votes
1
answer
3k
views
Innodb performs IO Operation when idle
I am currently doing some stress testing where I am pushing inserts to the database as fast as I can. The simulation inserts 10000 records per second. After inserting around 200 million records I ...
0
votes
2
answers
1k
views
Unreasonable performance expectations for RAID 0 performance?
This is my first time building a machine with a hardware RAID card. We bought a Dell T620 with the H710P RAID controller (1GB NV Cache), a 160GB Solid State Drive (SATA Read Intensive MLC 3Gbps), and ...
- 133
3
votes
4
answers
3k
views
Azure IO performance
We have a number of Windows Server virtual machines that we use for .NET and Java development and QA. I find Azure attractive for the obvious reasons but mainly for the opportunity to reduce non-...
- 131
1
vote
1
answer
1k
views
can Iowait cause significant RAM consumption increase
I have a node.js process running continuously on a linux EC2 instance, it extracts data and stores it in mysql. Some reads are also performed on the database.
The process can run well during hours or ...
- 121
0
votes
1
answer
9k
views
htcacheclean doesn't seem to be cleaning the cache [duplicate]
I'm running mod_cache with apache 2 on a fairly high traffic'd site.
I've set the htcacheclean to run in demon mode every 10 minutes to keep the cache at 4GB.
It's currently at 15GB and doesn't seem ...
- 362
4
votes
0
answers
2k
views
Why are pgpgin/s and majflt/s inconsistent in sar -B?
In sar -B ouptut, I would naively expect pgpgin/s to equal majflt/s multiplied by the system page size. From man sar:
pgpgin/s
Total number of kilobytes the system paged in ...
- 141