Free Republic
Browse · Search
General/Chat
Topics · Post Article

Skip to comments.

Bug in backup software results in loss of 77 terabytes of research data at Kyoto University
TechXplore ^ | 4 January 2022 | Bob Yirka

Posted on 01/04/2022 3:29:08 PM PST by ShadowAce

Computer maintenance workers at Kyoto University have announced that due to an apparent bug in software used to back up research data, researchers using the University's Hewlett-Packard Cray computing system, called Lustre, have lost approximately 77 terabytes of data. The team at the University's Institute for Information Management and Communication posted a Failure Information page detailing what is known so far about the data loss.

The team, with the University's Information Department Information Infrastructure Division, Supercomputing, reported that files in the /LARGEO (on the DataDirect ExaScaler storage system) were lost during a system backup procedure. Some in the press have suggested that the problem arose from a faulty script that was supposed to delete only old, unneeded log files. The team noted that it was originally thought that approximately 100TB of files had been lost, but that number has since been pared down to 77TB. They note also that the failure occurred on December 16 between the hours of 5:50 and 7pm. Affected users were immediately notified via emails. The team further notes that approximately 34 million files were lost and that the files lost belonged to 14 known research groups. The team did not release information related to the names of the research groups or what sort of research they were conducting. They did note data from another four groups appears to be restorable. Also unclear is whether the research groups who lost their data will be reimbursed for the money spent conducting research on the university's supercomputer system. Such costs are notoriously high, running into the hundreds of dollars per hour of computing time.

Some news outlets are reporting that the backup system was supplied by Hewlett-Packard and that the failure occurred after an HP software update. The same outlets are also reporting that HP has accepted blame for the data loss and is offering to make amends. The team at the university reported that the backup procedure was halted as soon as it became clear that something was awry and university officials suggest that in the future, incremental backup procedures will always be used to prevent the loss of data.


TOPICS: Computers/Internet
KEYWORDS: filesystem
Navigation: use the links below to view more comments.
first previous 1-2021-4041-52 last
To: ShadowAce

Must have had some evidence against Pfizer or J&J in it.


41 posted on 01/04/2022 8:09:59 PM PST by backwoods-engineer (But what do I know? I'm just a backwoods engineer.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

WTF is a Hewlett-Packard Cray computing system?

https://www.google.com/search?q=”Hewlett-Packard+Cray+computing+system”


42 posted on 01/04/2022 8:33:31 PM PST by catnipman (Cat Nipman: Vote Republican in 2012 and only be called racist one more time!)
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

That is a lot of naked Japanese women they downloaded.


43 posted on 01/04/2022 11:28:39 PM PST by minnesota_bound (I need more money. )
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

Should’ve been running LINUX.๐Ÿ˜


44 posted on 01/04/2022 11:42:53 PM PST by BiteYourSelf ( Earth first we'll strip mine the other planets later.)
[ Post Reply | Private Reply | To 4 | View Replies]

To: martin_fierro

This.๐Ÿ˜…๐Ÿ™„


45 posted on 01/04/2022 11:43:54 PM PST by BiteYourSelf ( Earth first we'll strip mine the other planets later.)
[ Post Reply | Private Reply | To 14 | View Replies]

To: ShadowAce

I use Bvckup 2 Pro $49.99
https://www.bvckup2.com

I can backup to multiple drives and online. It does incremental backups so is very quick and they are the actual files and are not zipped or encrypted by some proprietorial file system. You can open your files just like you do on your ‘c’ drive so you can copy them back.

I use another program to make a image file of windows and my files from time to time. Never trust a single backup or program.

Very fast. Once the initial files are backed up it does incremental files so even faster.
You can set the time to what you want for it to automatically backup.

File> Add new backup... Point to the location of the files you want to backup and where you want them to go and give it a description. It will backup even if you have a program open like the web browser or email.
I added my desktop files and bookmarks and email.
I have used other programs that were to complicated.
This program is super easy to use. Example file path below.

C:\Users\your name\Desktop\
E:\8TB Backup\Bvckup2\
Backup to E drive Desktop files

C:\Users\your name\AppData\Roaming\Mozilla\
E:\8TB Backup\Bvckup2\Mozilla backup\
Backup to E drive Firefox bookmarks

C:\Users\your name\AppData\Roaming\Thunderbird\
E:\8TB Backup\Bvckup2\Thunderbird backup\
Backup to E drive Thunderbird email


46 posted on 01/04/2022 11:44:04 PM PST by minnesota_bound (I need more money. )
[ Post Reply | Private Reply | To 1 | View Replies]

To: minnesota_bound

I have a 6TB NAS that I back up to and then that gets backed up to a cloud service.


47 posted on 01/05/2022 2:03:11 AM PST by AbolishCSEU (Amount of "child" support paid is inversely proportionate to mother's actual parenting of children)
[ Post Reply | Private Reply | To 46 | View Replies]

To: kiryandil

Serious IT departments are NOT using 12TB drives costing $400 each, just like serious concert sound reinforcement companies are not using home stereos.


48 posted on 01/05/2022 8:30:47 AM PST by BlueMondaySkipper (Involuntarily subsidizing the parasite class since 1981)
[ Post Reply | Private Reply | To 33 | View Replies]

To: BlueMondaySkipper
Right.

It was an example.

I was in charge of a serious IT department's backup strategy back in the day - these chumps that dumped upwards of 80Tb of data were more along the lines of a junior high school's computer lab backup strategy...

49 posted on 01/05/2022 8:37:57 AM PST by kiryandil (China Joe and Paycheck Hunter - the Chink in America's defenses)
[ Post Reply | Private Reply | To 48 | View Replies]

To: ShadowAce
find . -type f -mtime +30 | xargs -n 50 rm -f

You're welcome! (fast too)

50 posted on 01/05/2022 3:46:01 PM PST by The Duke (Search for 'Sydney Ducks' and understand what is needed.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: kiryandil

โ€œ I was in charge of a serious IT department’s backup strategy back in the day - these chumps that dumped upwards of 80Tb of data were more along the lines of a junior high school’s computer lab backup strategy...โ€

As was I. Home use quality drives may have the physical space the accommodate the data, but they most certainly donโ€™t have the throughput, duty cycle, or redundancy to do this at a commercial level.


51 posted on 01/06/2022 8:59:48 AM PST by BlueMondaySkipper (Involuntarily subsidizing the parasite class since 1981)
[ Post Reply | Private Reply | To 49 | View Replies]

To: BlueMondaySkipper
As was I. Home use quality drives may have the physical space the accommodate the data, but they most certainly donโ€™t have the throughput, duty cycle, or redundancy to do this at a commercial level.

That's true.

How much do you think those researchers would give for 7 of those crummy lightweight homes drives filled with their data, though? ย ย winking face

52 posted on 01/06/2022 9:18:36 AM PST by kiryandil (China Joe and Paycheck Hunter - the Chink in America's defenses)
[ Post Reply | Private Reply | To 51 | View Replies]


Navigation: use the links below to view more comments.
first previous 1-2021-4041-52 last

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
General/Chat
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson