Questions tagged [tar]
GNU tar creates and manipulates archives which are actually collections of many other files; the program provides users with an organized and systematic method for controlling a large amount of data.
348
questions
2
votes
1
answer
176
views
tar: Archive is compressed. Use -z option. Why tar asks me to use -z option when it is ignored in extract mode?
I get this error tar: Archive is compressed. Use -z option when trying to build my docker image:
RUN wget https://github.com/wal-g/wal-g/releases/download/v2.0.1/wal-g-pg-ubuntu-20.04-amd64.tar.gz -O -...
- 216
0
votes
0
answers
43
views
Saving contents of /proc/net
I'm trying to archive files from /proc/net when a process crashes. This is to aid in debugging, to get understanding about the machine status when crash happened.
The contents are not regular files, ...
- 333
0
votes
0
answers
97
views
How to force tar to include certain directories when using --exclude option
I need to backup a hierarchy. I need to keep the image as small and fast as possible. I use
tar -c --exclude="..."
with several --exclude=".." patterns to omit quite a few ...
- 101
0
votes
1
answer
151
views
Extract specific file from tar.gz file?
I am trying to extract a specific file from a tar.gz file. This is the code I used:
tar -zxvf Laura.tar.gz Laura/2021-10-22_16.52.10_pWOX5-YFP_x_membraneRed_Movie1/DC_CROPPED_h5/DC_CROPPED.h5
Laura....
0
votes
1
answer
168
views
How to compress a directory, using tar, and then delete the directory it compressed?
On my server there is an application directory that I need to make a full copy of it but the copy takes up a lot of space, I need to compress it and then remove it.
I already use the command
tar -czf ...
0
votes
3
answers
186
views
Tar backup command for exclude backup files & cache & include only httpdocs subdirectories
I am looking for a tar command that meets the following requirements:
I want to exclude all files ending with: backwpup & .zip (in between and before can be any e.g. uploads/123backwpup-123.zip)
I ...
- 103
0
votes
1
answer
51
views
Debian: Simple way to compress files wanted
I often need to compress archives in Linux. is there a simpler way instead of always building such complicated commands?
tar zcvf /tmp/mybackup.tar.gz /home/important
- 615
2
votes
2
answers
135
views
Make a gzip-compressed tar archive of files and directories that have a specific letter
I want ONLY directories and files that contain the letter.
I've been able to recreate it from a previous question. But the tar is not a gzip.
And when trying to make it a gzip it...doesn't remove ...
- 23
0
votes
0
answers
2k
views
is there a faster way to extract a large file tar in lz4 format?
I have a 7 TB file that is tared with lz4 compression. It takes about 8 hours to extract. is there a faster way to do this?
This is the command I use:
lz4 -d /mnt/tmp/7TBFile.lz4 | tar -xvf -
0
votes
3
answers
174
views
Magnetic tape alternative [closed]
I have recently been assigned as a systems administrator to look after a streaming service and its related server architecture.
Once a week i travel 3hours to turn around a magnetic tape library which ...
- 117
1
vote
0
answers
423
views
Extract archive with tar but skip unchanged files
I have a nightly process that unarchives a roughly 40 gigabytes large tar.gz file like this:
tar -xzf latest-backup.tar.gz
This step takes about 10 minutes, although often only a few files have ...
- 165
0
votes
1
answer
255
views
Tar and 7z compression on Linux - what's the difference?
I have a problem!
I have a backup script in python. It backups all the folders in /var/www/ into different .tar.7z for each folder inside /var/www/
The problem is that the compression time is very ...
- 339
1
vote
1
answer
83
views
What behavior has tar on TERM signal while doing incremental backup?
I've written a backup script in python using tar (via subprocess) to do incremental backups of my files. As some of the full backups are rather big (like my picture folder) and take multiple hours to ...
- 157
0
votes
2
answers
1k
views
Can we use make tar multi-threaded without any sort of compression
I have to backup 500GB using the traditional tar -cvf command (without any compressions)
However tar is adding each file one by one and this way it's taking a very long time
Is there any way to make ...
- 148
1
vote
0
answers
2k
views
tar -xvf error "Can't restore time" when trying to extract in certain directories on mac osx but works in others
Strangest thing.
I downloaded the openjdk release here: https://jdk.java.net/16/
When running tar -xvf on the file in the ~/Downloads directory it works fine.
But when running in /tmp or another ...
- 183
3
votes
1
answer
218
views
Tar extracting from multi-volume tape whilst computing shasums
As part of our backup system, we replicate zfs datasets from a TrueNAS system to a couple of backup servers, one of which is running TrueNAS Scale and has a LTO-5 tape drive connected. We occasionally ...
- 41
0
votes
1
answer
301
views
Execute following steps after rsync is finished
First I thought that rsync isn't syncing everything described here Limitations of rsync? Not syncing everything
Now after a day I found out that it was actually still working/ running and now all the ...
- 113
1
vote
1
answer
42
views
Validate tar extraction
I have a script that processes large tar.bz2 and tar.gz files.
So the user doesn't think it has locked up they are piped in from pv:
pv redacted.tar.bz2 | tar -xj
pv redacted2.tar.gz | tar -xz
...
- 181
5
votes
1
answer
491
views
Unable to compress 18GB file on Ubuntu Linux 18.04
It never happened to me before, but I'm unable to do a simple task such as compressing a 18.5 GB file on Ubuntu Linux 18.04 with any of the popular compressing tools such as gzip, bzip2 and 7z. All of ...
- 181
0
votes
1
answer
322
views
Renaming a tar file in unix script
cd /var/opt/sw/e4/data/dev/e4_dev/DEVL3/EW/EWD1/DATA/AED/INPUT
for file in *;do tar -czf ../OUTPUT/"${file}".tar; done
According to the above code the files in input path eg. aedlog, aaalog....
1
vote
2
answers
12k
views
Creating backups with tar.gz and rsync
I'm trying to use this bash script to create and send the tar.gz file to my backup server, some reason it doesn't seem to copy it, is there a better solution to what I'm trying to do?
I'd rather have ...
- 13
3
votes
1
answer
3k
views
How do untar as the current user by default?
I have a batch of CentOS EC2 instances, which have an additional user added to them via ansible. Normally when unpacking a tarball, the owner is centos.centos (default login for EC2 Centos). When an ...
- 31
1
vote
2
answers
1k
views
How can I transfer a dedicated bare metal server into a VM?
I have the following problem: We have a dedicated (bare metal) hardware server (Debian 10) on which we have no direct physical access. Now I want to transfer all data and applications that are on this ...
- 6,209
0
votes
0
answers
53
views
Diff a tarball on stdin against a directory hierarchy?
I have a multi-terabyte filesystem I want to diff against a multi-terabyte tar file. The tar file is only available from stdin - no seeking allowed. I do not have the disk space to write the tar on ...
- 155
1
vote
1
answer
2k
views
How to owerwrite directory with tar?
I want to extract files with tar so that the whole directory will be owerwritten.
I added --overwrite-dir but still old files in test dir remains.
For the test, I have /home/ubuntu/f1/test/first.log ...
- 183
2
votes
1
answer
723
views
Fastest way to tranfer data over the network between two servers
I have two storage servers at home - one is my workstation, which has a pair of 3TB disks in RAID1.
At first, I stored all of my data there. Recently, I received a small SBC with four 1TB disks.
I set ...
- 155
0
votes
2
answers
107
views
RHEL 7 security patch installation from tar
Novice Linux guy here
I have a huge (5G) linux-security-patches tar. Upon extracting, I see its got loads of rpms. What is the most efficient way to install them all ? Also, is it same as installing ...
- 103
1
vote
1
answer
174
views
How to take backups resilient to data corruption , fast and reliable on Linux? [closed]
I need a resilient backup solution. I was using tar with pigz, however I am afraid of tar.gz corruption and losing data. Projects are between 4TB - 25TB of data for each project folder. We are talking ...
- 536
0
votes
1
answer
243
views
Tar directory is new error
I am trying to make a backup of two directories, /etc/httpd and /etc/letsencrypt
I am using the command
tar Pcfvg httpd_backup.tar /etc/httpd/ /etc/letsencrypt/ >/dev/null
The redirect to /dev/...
- 195
0
votes
1
answer
53
views
Does tar create point-in-time snapshots?
I know that tar can be used to create a backup.
However I wonder if the operation creates point-in-time backups or not.
Basically I wonder if it grants consistency if the directory and files are ...
- 264
0
votes
0
answers
41
views
Error while trying to backup site files
# tar -czf /backup/backup.tar.gz /var/www/html
tar: Removing leading `/' from member names
tar: /var/www/html/wp-content/uploads/2020/02: file changed as we read it
# ls -la
total 17160456
drwxr-xr-x ...
- 19
4
votes
2
answers
2k
views
Hash files in tar file
I have two *.tar files with similar contents. I want to verify which files are the same. A lot of the files are big so I comparing hashes would require extracting every file from each tar and ...
- 193
0
votes
0
answers
161
views
tar | pigz dies if run under cron
I run a kind of backup script using tar joint with pigz for compression.
The problem is that
tar -zcf /file.tar.gz /folder
runs under cron.
While
tar -I pigz -cvf /file.tar.gz /folder
nor
tar ...
- 151
2
votes
1
answer
262
views
Best strategy for daily back up of terabytes of data with millions of files
Currently I create an NFS and mount it on a special server that does the actual backups and uploads it to s3. My main concern is I feel like overcomplicated this simple task. I schedule cron job to ...
- 164
1
vote
1
answer
2k
views
Why tar flag --strip-components ignored sometimes?
I have an automated docker images build where I download elasticsearch archive and extract it via:
tar zxf archive.tar.gz --strip-components=1 -C /path/to/dir
And it always worked until the latest ...
- 243
0
votes
1
answer
923
views
How to exclude a folder by name recursively when making a tar archive?
I know it is possible to exclude a particular folder by a command like this:
tar --exclude='/srv/www/project/node_modules' -zcvf /backup/project.tgz .
My question how to exclude all folders named ...
- 214
3
votes
1
answer
5k
views
Backup to remote server and create tar on the fly
I would like to backup a directory from my production server to a backup server using rsync.
My current solution:
tar directory
on production server send to remote server via rsync
My Problem with ...
- 133
2
votes
2
answers
1k
views
Extracting single file from large tar.gz file
I have an incredibly large tarball. I'd extract several files out of many thousands within the archive. I'm on CentOS 6.10 running GPFS 4.2.3. I've seen from this answer that pigz is useful in ...
0
votes
1
answer
39
views
Command/script to extract "sub-tarball" from tarball
I am aware of that I can extract single/multiple files from a tar archive, but is there a command that I can easily pass to immediately create a "sub-tarball" from an existing tar archive? This is ...
- 13
0
votes
2
answers
629
views
How to untar a very large file with missing space
I have reinstalled my server and before that i did a backed up.
My backup is a tar file without special compress option (meaning that the size of the tar file is approximately the same than the size ...
0
votes
1
answer
581
views
abnormal very high CPU during lengthy write operations
I have a set of large files which I sometimes copy back and forth between a Linux box and a Windows box. The files are each about 2 GB, and there tend to be 10 or so of them (they're a VM image). ...
- 145
0
votes
1
answer
377
views
Uploading files using tar
Can you help me?
I have this piece of code that I'm using on my Ubuntu 16.04 server
tar -c /var/www/dxa/backup.zip | ssh <my-server-ip> -p 8000 'tar -xvf - -C /'
But everytime I run this ...
- 23
0
votes
1
answer
1k
views
ubuntu server tar -xvf failed for solr.tar.gz
I am trying to extract solr server downloaded from https://www.apache.org/dyn/closer.lua/lucene/solr/7.7.2/solr-7.7.2.tgz
But am getting an error on ubuntu server. gzip: stdin: not in gzip format
I ...
- 157
1
vote
1
answer
1k
views
Backup a whole server over the network using tar
I have a server that is 80G total size and has no more space so i cannot backup on the server. I found this command
ssh server2 "tar -zc -C /srcdir ./path" | tar -zx -C /destination
but it is not ...
- 143
0
votes
0
answers
15
views
Tar is not Finishing When Running as Cronjob [duplicate]
My goal is to regularly pack the most important files on a server into a tar.gz file. In order to automate this I am using a cronjob to execute a script:
$ crontab -l
# blablabla
* * * * * /home/...
- 101
0
votes
0
answers
170
views
resume tar command when nohup/screen is not possible
I have a dataset which is quite big - a few petabytes.
Untarring it into a new filesystem takes a couple days. The problem is that the server does not allow me to have a process running over 24h - it ...
3
votes
1
answer
4k
views
NFS - Very slow performance with small files (Also with CIFS)
I am having very bad performance with NFS between two debian stretch machines, and I can't find a solution.
I am using rsync at the server side in the exports file and tcp,hard,intr,rsize=32768,wsize=...
- 207
1
vote
1
answer
865
views
Tar archive error message
I am getting following error while archiving the user's homedirectory.
(trying to migrate a user from one macbook to other since the user's keys are associated with AWS and other tools)
tar cfv ...
- 135
0
votes
1
answer
1k
views
how to create a tar file with checksums, but without compression
(note, this is not a duplicate of Creating a tar file with checksums included)
I'm familiar with using tar + gzip to create a compressed tar file (tar cf - files | gzip > something.tar.gz), and ...
- 145
1
vote
1
answer
2k
views
This does not look like a tar archive
I'm trying to fast copy some big directories from my remote Debian 9 server to my local Debian 9 on WSL (Windows Subsystem for linux; Windows 10 Pro) with ssh "tar" | tar pipe.
ssh [email protected] "...
- 365