Free Republic
Browse · Search
General/Chat
Topics · Post Article

Skip to comments.

Tech help needed
n/a ^ | 8/2/04 | self

Posted on 08/02/2004 7:42:09 PM PDT by sharktrager

OK...I need help.

I have a HUGE (320MB) text file I need to pare down to a workable size. It is a government data dump so, of course, it has a ton of useless data burying the data I need.

What I need to do first, and foremost, is just get rid of the dang quotation marks filling empty fields.

Here's the problem. No matter what program I open it with, it puts so much demand on my PC that I can barely load the thing, let alone start the clean-up.

As for the obvious things:

1) No, they can't resend it in a few smaller files. They are the government and they are not here to help me.

2) I can't break it apart myself. If I could cut and paste that much the replacement I am trying wouldn't be a problem.

Sooooo...any ideas? I already have 512MB memory, and have tried ramping up the virtual memory, and it is a no go.

Anyone have any ideas?


TOPICS: Computers/Internet
KEYWORDS: huge; pia; textfiles

1 posted on 08/02/2004 7:42:15 PM PDT by sharktrager
[ Post Reply | Private Reply | View Replies]

To: sharktrager

A simple bash shell script should work fairly well.


2 posted on 08/02/2004 7:53:55 PM PDT by ShadowAce (Linux -- The Ultimate Windows Service Pack)
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

I bet you could hear the "huh?" all the way there.


3 posted on 08/02/2004 8:01:05 PM PDT by sharktrager (The road to hell is paved with good intentions. And the paving contractor lives in Chappaqua.)
[ Post Reply | Private Reply | To 2 | View Replies]

To: sharktrager
File Splitter

What is The File Splitter?
The File Splitter is a free Windows program that will split any file into pieces of any size and later combine the pieces to form the original file. This could prove useful in a number of situations:
  • Suppose you need to transfer a file from one stand-alone computer to another. A floppy disk is often the only possible means for transferring the file, but if the file is larger than 1.44 megabytes, it won't fit on a single floppy. Simply split the file onto floppy disks, and then copy the pieces from the floppy disks to the other computer and easily reconstruct the original file.

  • Many email systems place a limit on the size of an email message. What if you want to email someone a file that is larger than the maximum size? Split the file into pieces smaller than the limit and email the individual pieces.

  • There are many other possibilities for using The File Splitter. I'm sure you can think of a situation where The File Splitter would be helpful.

4 posted on 08/02/2004 8:07:08 PM PDT by John Jorsett
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce
I was thinking about a two line Perl script to break it into managable chunks. Or use VI to open it.

/john

5 posted on 08/02/2004 8:25:30 PM PDT by JRandomFreeper (Maybe I am a geek again, but I can cook! No discuta conmigo.)
[ Post Reply | Private Reply | To 2 | View Replies]

To: John Jorsett

That did the trick.

Thanks.


6 posted on 08/02/2004 11:27:56 PM PDT by sharktrager (The road to hell is paved with good intentions. And the paving contractor lives in Chappaqua.)
[ Post Reply | Private Reply | To 4 | View Replies]

To: sharktrager

No problem.


7 posted on 08/03/2004 5:53:39 AM PDT by John Jorsett
[ Post Reply | Private Reply | To 6 | View Replies]

To: sharktrager

Download a freeware file splitter/joiner program and split it into smaller chunks at the file level. Open each one separately and edit it then save it. Then use the file splitter/joiner to join them back up.


8 posted on 08/03/2004 9:34:59 AM PDT by Bloody Sam Roberts (May the wings of Liberty never lose so much as a feather.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: sharktrager

Ooops. I see you already found one.


9 posted on 08/03/2004 9:36:20 AM PDT by Bloody Sam Roberts (May the wings of Liberty never lose so much as a feather.)
[ Post Reply | Private Reply | To 1 | View Replies]

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
General/Chat
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson