Questions tagged [synchronization]
The synchronization tag has no usage guidance.
490
questions
61
votes
8
answers
45k
views
Rsync : copying over timestamps only
Currently I have two directories A/ and B/ which are identical in every respect, with the exception of the timestamps. Therefore if I run the command :
rsync --dry-run -crvv A/ B/
then all files ...
- 1,009
55
votes
9
answers
144k
views
How to interrupt software raid resync?
I want to interrupt a running resync operation on a debian squeeze software raid. (This is the regular scheduled compare resync. The raid array is still clean in such a case. Do not confuse this with ...
- 551
33
votes
2
answers
21k
views
Copy directory tree without empty directories?
I have the following tree
# upper letters = directory
# lower letters = files
A
|-- B
|-- C
|-- D
|-- e <= file
|-- F
|-- G
I need to copy this tree to another destination, ...
- 1,289
33
votes
3
answers
24k
views
How does dropbox version/upload large files? [closed]
I have a free dropbox account (2GB), and I was wondering how the versioning of large files works.
I have a full backup of all my webfiles that sites @ just over 1GB. After the initial upload of 1GB, ...
- 760
32
votes
5
answers
91k
views
How to upload a directory recursively to an FTP server by just using ftp or lftp?
I am writing a Linux shell script to copy a local directory to a remote server (removing any existing files).
Local server: ftp and lftp commands are available, no ncftp or any graphical tools.
...
- 1,334
29
votes
7
answers
87k
views
Is it possible to sync two linux directories in real time ?
I would like to sync a directory of an internal hard drive to an external hard drive.
When I write or modify a file on my internal hard drive, automatically the same file would be copied to the ...
- 931
28
votes
2
answers
50k
views
rsync : transferring files in one direction only
Is it possible to use rsync to copy files in one direction only?
For example, suppose we have:
left/a.txt
right/a.txt
where the files are initially identical.
If one then modifies right/a.txt, ...
- 1,009
27
votes
7
answers
12k
views
Options to efficiently synchronize 1 million files with remote servers?
At a company I work for we have such a thing called "playlists" which are small files ~100-300 bytes each. There's about a million of them. About 100,000 of them get changed every hour. These ...
- 383
26
votes
2
answers
11k
views
What's the advantage of synchronizing UID/GID across Linux machines?
Before I plunge into the depths of how to synchronize UID's/GID's across my different Linux machines, I would like to know what is actually the benefit?
I know that this keeps file synchronization ...
- 417
24
votes
4
answers
5k
views
Pushing Large Files to 500+ Computers
I work with a team to manage 500-600 rented Windows 7 computers for an annual conference. We have a large amount of data that needs to be synced to these computers, up to 1 TiB. The computers are ...
- 340
24
votes
6
answers
10k
views
Bidirectional real-time sync of large file tree between two distant linux servers
By large file tree I mean about 200k files, and growing all the time. A relatively small number of files are being changed in any given hour though.
By bidirectional I mean that changes may occur on ...
- 471
20
votes
5
answers
21k
views
Is execution of sync(8) still required before shutting down linux?
I still see people recommend use of sync; sync; sync; sleep 30; halt incantations when talking about shutting down or rebooting Linux.
I've been running Linux since its inception and although this ...
- 839
19
votes
6
answers
51k
views
Faster rsync of huge directory which was not changed
We use rsync to backup servers.
Unfortunately the network to some servers is slow.
It takes up to five minutes for rsync to detect, that nothing has changed in huge directories. These huge ...
- 3,603
19
votes
2
answers
12k
views
How can I *include* folders through Dropbox's selective sync feature in Linux, from the command line?
After another user asked this question: Dropbox on linux server - how to include/exclude folders?
My icon is still not working after performing some of the steps listed in other questions and answers:
...
- 293
18
votes
10
answers
5k
views
Password Manager that allows syncing accross platforms [closed]
I use OS X, Linux, Solaris and windows for work and from home. There are good tools that allow me to manage the many logins/passwords required platform independently. But mostly they expect me to ...
- 920
15
votes
5
answers
10k
views
Synchronizing very large folder structures
We have a folder structure on our intranet which contains around 800,000 files divvied up into around 4,000 folders. We need to synchronize this to a small cluster of machines in our DMZs. The depth ...
- 251
13
votes
2
answers
54k
views
Correcting CentOS System Clock
Using the time C function (number of seconds since the Epoch) shows that the time on my current CentOS 6 server is about 7 hours behind compared to another server with the correct time. How can I ...
- 337
13
votes
1
answer
783
views
Does rsync change file content (dealing with hardlinks)?
Imagine I have a file remote/A which syncs to local/A and local/B is a hardlink of local/A. Is there a threat that on a next remote/A change and synchronization rsync will not delete local/A but only ...
- 232
10
votes
11
answers
76k
views
SyncToy alternative for Windows 7 - synchronise folder with network drive
I have a Windows 7 laptop, which I want to back up to a network folder.
There is a drive (partition) on my laptop machine that I want to backup to a network
drive. If I delete a file in the folder on ...
- 1,720
9
votes
3
answers
16k
views
How to keep the time on multiple servers in sync
I want a cluster of servers to have the exact same time.
All are using UTC time zone.
Using NTP currently, i still see they seem to be off by 10-20 miliseconds each.
Whats the best way to have them ...
- 2,457
8
votes
4
answers
19k
views
Syncing user accounts across multiple servers
I have been wondering if there is a better way to manage user accounts across multiple servers.
At the moment, I have a main user account (e.g. 'user') on my home/dev machine, and for each server I ...
- 183
7
votes
1
answer
16k
views
Access Denied when syncing between s3 buckets on different AWS accounts
I'm trying to setup sync between two buckets on different AWS accounts.
I got cp working with:
@ubuntu:~$ s3cmd cp -v s3://src/dir/ s3://dest/folder --recursive
I am the user who owns /src/dir and ...
- 182
7
votes
3
answers
10k
views
OpenLdap synchronize home directory between client and server
I have recently setup Ubuntu ldap client authentication on karmic and lucid desktop clients and the ldap server is running on an 10.04 LTS server and this is the link I followed for client setup and ...
- 1,723
7
votes
5
answers
1k
views
Synchronize Chrome bookmarks across multiple machines [closed]
Update: no longer relevant, please vote to close.
We all know and love xmarks for keeping Firefox bookmarks sync'd across computers, but...
Almost everyone at our company has switched to Chrome. Has ...
- 5,271
7
votes
3
answers
5k
views
Is there any way to synchronize AD users with Office 365 but still be able to edit them online?
I'm performing a migration to Office 365 from a third-party mail server (MDaemon); the local Active Directory doesn't include any Exchange server, and never had any.
We will need directory ...
- 70.5k
6
votes
3
answers
3k
views
What program to use for incremental backup of large single file
I'm looking for a program to make incremental backups of one large file (3GB) on windows systems over internet.
Any idea? Thanks!
- 65
6
votes
5
answers
809
views
Is there a free tool that syncs files in both directions, on a schedule?
We have a directory of files (about 4GB) - and we'd like to synchronise between 3 locations across the internet. However, we're very conscious about bandwidth, so we need the following features:
...
- 5,126
6
votes
2
answers
11k
views
Getting contact list from IMAP server
I want to update my local Thunderbird address book with a list of valid email addresses on an IMAP server. I'm not that familiar with IMAP, does it allow this? If so, could someone point me to an ...
- 270
6
votes
2
answers
123
views
Problems with Azure AD connect custom install
I badly need your clever advice!
The thing is I am trying to extend on-premises active directory to Azure.
And one of my coworkers mentioned that if it’s over 100,000 items then you’ll want to ...
- 606
6
votes
1
answer
1k
views
Is there a global config file for Unison synchronization tool?
I'd like all my unison profiles to have
merge = Name * -> meld CURRENT1 CURRENT2
diff = /usr/bin/meld CURRENT1 CURRENT2
confirmmerge = true
as options. I know I can add these options to each .prf ...
- 197
6
votes
1
answer
2k
views
Google Cloud Storage Fuse vs GlusterFS, pros, cons and costs
I'm setting up some web servers on Google Cloud Compute and I need a directory synced between all the servers to keep the web sites in sync.
I'd also like to use the autoscaling feature to provision ...
google-cloud-platform
6
votes
1
answer
4k
views
Need to make all the timestamps on local files match their corresponding ftp hosted file's timestamp
I screwed up and downloaded ~10GB worth of files from my offsite backup location (ftp server) without checking the 'preserve timestamp' option in the client.
The PC with the local files is a netbook ...
6
votes
4
answers
3k
views
Linux User Synchronization [closed]
What would you recommend as a way to manage users in a small Linux network, so that users only need to be created on one machine with them synchronized automatically on the others. Some are laptops ...
- 628
6
votes
10
answers
3k
views
Synchronize with intelligent move and rename - windows
I'm looking to synchronize two windows folders over a wireless network. I'm currently using Syncback, but it doesn't seem suitable for large reorganisations over a slow network.
i.e. if I move a ...
- 191
5
votes
7
answers
28k
views
Simple one-way synchronisation of user password list between servers
Using a RedHat-derivative distro (CentOS), I'd like to keep the list of regular users (UID over 500), and group (and shadow files) pushed to a backup server.
The sync is only one-way, from the main ...
- 519
5
votes
8
answers
7k
views
Suggestions for using Active Directory credentials (user name/password) with Google Apps?
I run a small college network with approximately 150 user accounts both in my Active Directory and in Google Apps. I'm looking to link AD with G-Apps. Currently my users are required to maintain ...
- 2,051
5
votes
1
answer
25k
views
w32tm /resync not working instantly
This Question is mostly for other people to learn from and to satisfy my own curiosity if someone knows why it does the following.
We had a problem with our DC not having the right time due to it ...
- 213
5
votes
2
answers
1k
views
Live file syncronization across multiple Linux servers with millions of files and directories
What is the best way to synchronize huge data of a running production server?
Our server has over 20 million files (small files with 10k and bigger files up to 50MB) stored in 1 millon directories. ...
- 439
5
votes
2
answers
13k
views
SYNC a folder on Ubuntu server with amazon S3 bucket automatically
I have an app running on Digital Ocean server and there are two folders which are being updated with user uploaded images.
Is it possible to transfer the new uploads from the server folder to s3 ...
- 161
5
votes
5
answers
551
views
Do you know of an online back up & sync solution for Windows that actually works?
I've been looking for some time for a program that backups your data to the cloud and is also able to sync 2 or more computers. By syncing I mean, if I change something on one computer that it should ...
user21316
5
votes
1
answer
877
views
Rsync over sshfs skips filenames starting with underscore
I'm trying to sync a directory tree from OS X (10.11) to Ubuntu 14.04. While most of the files get transferred just fine, files with names starting with _ (underscore) do not.
Here is the command I ...
- 191
5
votes
1
answer
2k
views
DRBD as DR: syncing datastores of 2 ESXI hosts, vmdk consistency?
does anyone have experience with using DRBD (protocol C) to sync parts of the datastores of 2 esxi hosts for disaster recovery of selected guests?
I have 2-3 guests that should be able to recover ...
- 51
5
votes
4
answers
747
views
Copying/syncing email from a live server over network
The scenario is as follows:
Copying and then syncing from a live mail server via network(only) to another server.
The mail server is live meaning that lots of files (mails) are being altered, ...
- 461
5
votes
1
answer
5k
views
Active Directory max photo size
We are looking at putting photos in Active Directory. We have many domain controllers, some are in areas with not a lot of bandwidth. What are the recommend/max files sizes for photos or is this a ...
- 244
5
votes
6
answers
983
views
Internal Dropbox system
Is there a solution out there for a windows environment that would provide a two way, centralized sync between PCs? We have several PCs in our lab on which we need to automatically sync files across, ...
- 6,573
5
votes
1
answer
2k
views
Master-Master File Sync between DataCenters
My problem is regarding a master-master (3 master nodes) file synchronization setup, where each node is in a different DataCenter. I have three application servers where users can create/modify/delete ...
- 221
5
votes
1
answer
7k
views
csync2 ERROR: Connection to remote host failed
I was unable to find any articles to answer this question, so my best bet was to post this here:
Scenario
We have 2x application servers in production hosting a PHP website and I would like some ...
- 51
5
votes
1
answer
2k
views
Forward-sync to HDFS? (OR continue an incomplete hdfs upload?)
Anyone have a good suggestion for doing a forward sync to HDFS? ("forward-sync" in contrast to "bi-directional sync")
Basically I have a large number of files I want to put into the HDFS. Its so ...
- 993
4
votes
7
answers
5k
views
Network File Synchronization Tools
We need to keep a set of files on multiple servers synchronized. This seems like a simple problem, but I'm not a full-time system administrator, and while I've found a simple solution, it's not very ...
- 262
4
votes
6
answers
9k
views
How to keep load-balanced servers synced even with deleted files?
I've recently setup a loadbalanced solution for our websites. We host about 200 sites, most run of our custom application, but some are running wordpress blogs (in which files can be uploaded/deleted)....
- 3,965