Wow, that's a lousy compression.

DEEMAN223344 · Post by **DEEMAN223344** » Wed Dec 29, 2010 10:23 pm

I compressed a setup file for my Hub project, and the .zip file was larger than the actual file I was compressing!

Muzozavr · Post by **Muzozavr** » Thu Dec 30, 2010 3:22 am

DEEMAN223344 wrote:I compressed a setup file for my Hub project, and the .zip file was larger than the actual file I was compressing!

LOL, that IS weird. Although all lossless compressions (including zip) will have some kinds of data that do become larger after "compressing".

Emerald141 · Post by **Emerald141** » Thu Dec 30, 2010 1:56 pm

Muzozavr · Post by **Muzozavr** » Thu Dec 30, 2010 7:46 pm

Emerald141 wrote:How does compression work, anyway? Do the individual bytes somehow become smaller?

Bytes are 8 bits each and that does not change. You meant "symbols", right?

Lossless compression works by rewriting the data in a way that allows the original to be restored, but it's smaller than the original.

General compression algorithms are usually fairly complicated, but the idea can be explained on two simple to understand methods: byte-pair encoding and run-length encoding. I'll demonstrate with symbols, it's easier to understand.

Byte-pair encoding:

Suppose we have a string that looks like this:

AABAACDBAEAAFAAGAAHAA

The "AA" pair appears more often than any other pair. So we replace it with a symbol that isn't yet there in the string. An additional sequence is written as a "table of replacement".

Z=AA
ZBZCDBAEZFZGZHZ

Not very effective here, but in longer examples it can sometimes work miracles. Still, it's fairly limited.

Run-length encoding:
Suppose we have a string like this:

DDDDrrrRRRddddddrrrRRuuuLrrrUULLL

That can be imagined as a solution for a box pushing puzzle, Up Down Left Right, big letters for pushing, small levels for just moving.

Now we do this:
4D3r3R6d3r2R3uL3r2U3L

In both cases, it's perfectly restorable but it is shorter... compression.

One other method is supposedly REALLY popular as something programs do AFTER applying their own methods... so it's kinda important for all file compressors, but... this one is more complicated. Maybe it's the fact that I'm ill right now, but... I don't really get it.

Anyway, it's here: http://en.wikipedia.org/wiki/Huffman_coding

DEEMAN223344 · Post by **DEEMAN223344** » Sun Jan 02, 2011 12:28 am

Another thing: WHAT IS MAKING THESE HUB PROJECTS SO HUGE!?
MOMII was only 86 MB compressed.
My new (and shorter) project is....
*drumroll*

THIRTY GIGABYTES, EVEN WHEN IN A .ZIP.
Strange thing is, MOMII BETA is not only longer than MOTLWL DEMO 1, it had a_lot_of unnecessary .exe files, while MOTLWL does not.
WHY IS IT SO GIGANTIC?!

Technos72 · Post by **Technos72** » Sun Jan 02, 2011 12:37 am

Yikes!

That's more GB than how much my old computer could hold!

DEEMAN223344 · Post by **DEEMAN223344** » Sun Jan 02, 2011 12:55 am

it is indeed large, that is a fact.
Why is it so big?

tyteen4a03 · Post by **tyteen4a03** » Sun Jan 02, 2011 3:03 am

What do you have inside it? List the biggest items may help.

EDIT: From what I saw, the huffman thing calculates the frequency of a letter in a document before compiling it into a proper 'archive'. If a letter, for example, 'a', existed so many times, the program replaces 'a' with a bit. The letter 'z' appeared the least, the program replaces 'z' with at most 25 bits (note: not 26 because of 0). Each letter, at usual, takes 1 byte (8 bits), but using huffman's thingy (xD), 'a' only takes 1 bit (1/8) while 'z' takes 25 (25/8 -> 3.studIo wait I mean something). This effictively selectively compress letters that appeared the most.

FUNFACT: Usually the most appeared 'letter' is not letters, but space.

Edited by tyteen: merged post~ *whistles away*

DEEMAN223344 · Post by **DEEMAN223344** » Sun Jan 02, 2011 3:29 am

As far as I know, the biggest things are the Hub, Player Profiles, Wonderland.exe, and wg.exe.

~xpr'd~ · Post by **~xpr'd~** » Sun Jan 02, 2011 3:35 am

That's a lot. You should try .7z or .rar. Maybe that'll be better?

Qloof234 · Post by **Qloof234** » Sun Jan 02, 2011 3:37 am

Quick question, why are the .exe's in the zip to begin with? Just wondering.

DEEMAN223344 · Post by **DEEMAN223344** » Sun Jan 02, 2011 3:41 am

Because I was too lazy to take them out.
It's the demo's .exe files anyway, and you HAVE TO know about MS's site to find this forum anyway (also why the editor is free, but only here).

Guppy Star · Post by **Guppy Star** » Sun Jan 02, 2011 3:46 am

DEEMAN223344 wrote:Another thing: WHAT IS MAKING THESE HUB PROJECTS SO HUGE!?
MOMII was only 86 MB compressed.
My new (and shorter) project is....
*drumroll*

THIRTY GIGABYTES, EVEN WHEN IN A .ZIP.
Strange thing is, MOMII BETA is not only longer than MOTLWL DEMO 1, it had a_lot_of unnecessary .exe files, while MOTLWL does not.
WHY IS IT SO GIGANTIC?!

You can try compressing all wdf by editing in pic manager, saving it as JPG and renaming it to wdf (first delete or move original to some location)
You can also use FASTEST option while compressing using WINRAR etc

~xpr'd~ · Post by **~xpr'd~** » Sun Jan 02, 2011 4:09 am

Guppy Star wrote:You can try compressing all wdf by editing in pic manager, saving it as JPG and renaming it to wdf (first delete or move original to some location)
You can also use FASTEST option while compressing using WINRAR etc

the first one probably won't work. the picture managers i use save jpg's at full quality, uncompressed. plus, jpg's aren't good for compression.
using the fastest option while compressing will make the file bigger. i don't see why anyone would do that.

tyteen4a03 · Post by **tyteen4a03** » Sun Jan 02, 2011 4:53 am

Tip to all hub packagers: Only include files that were changed/added in your package. Always make notes of what files you edited in each stable revision.

A revisioning system could do it for you, unfortunately they don't really support binary files well.

Guppy Star · Post by **Guppy Star** » Sun Jan 02, 2011 12:37 pm

~xpr'd~ wrote:
Guppy Star wrote:You can try compressing all wdf by editing in pic manager, saving it as JPG and renaming it to wdf (first delete or move original to some location)
You can also use FASTEST option while compressing using WINRAR etc
the first one probably won't work. the picture managers i use save jpg's at full quality, uncompressed. plus, jpg's aren't good for compression.
using the fastest option while compressing will make the file bigger. i don't see why anyone would do that.

Depends on the file you use
If there is only one file of large size, the RAR/ZIP size increases
If there are lots of small files of total 100MB size perhaps, the fastest option compresses it up to 60 above MB

EDIT: 60 MB or a little above - made a mistake earlier

Edited by tyteen: merged post

Muzozavr · Post by **Muzozavr** » Sun Jan 02, 2011 3:43 pm

DEEMAN223344 wrote:Another thing: WHAT IS MAKING THESE HUB PROJECTS SO HUGE!?
MOMII was only 86 MB compressed.
My new (and shorter) project is....
*drumroll*

THIRTY GIGABYTES, EVEN WHEN IN A .ZIP.

DEEMAN223344 wrote:Strange thing is, MOMII BETA is not only longer than MOTLWL DEMO 1, it had a_lot_of unnecessary .exe files, while MOTLWL does not.
WHY IS IT SO GIGANTIC?!

DEEMAN223344 wrote:DEMO 1

*imagines the size of the full project*

DEEMAN223344 · Post by **DEEMAN223344** » Sun Jan 02, 2011 4:05 pm

Muzozavr wrote:
DEEMAN223344 wrote:Another thing: WHAT IS MAKING THESE HUB PROJECTS SO HUGE!?
MOMII was only 86 MB compressed.
My new (and shorter) project is....
*drumroll*

THIRTY GIGABYTES, EVEN WHEN IN A .ZIP.

DEEMAN223344 wrote:Strange thing is, MOMII BETA is not only longer than MOTLWL DEMO 1, it had a_lot_of unnecessary .exe files, while MOTLWL does not.
WHY IS IT SO GIGANTIC?!

DEEMAN223344 wrote:DEMO 1

DEEMAN223344 wrote:DEMO 1
*imagines the size of the full project*

^
With what I'm planning, it may be OVER NINE THOUSAND GB!!!!!!
=9.01 Terabytes.

Qloof234 · Post by **Qloof234** » Sun Jan 02, 2011 4:10 pm

Just to make sure, are you sure you're reading it correctly? WA and MoFI combined isn't even one GB to begin with.

DEEMAN223344 · Post by **DEEMAN223344** » Sun Jan 02, 2011 4:17 pm

OH! It's thirty MB, not GB. Silly me.

Muzozavr · Post by **Muzozavr** » Sun Jan 02, 2011 4:58 pm

DEEMAN223344 wrote:OH! It's thirty MB, not GB. Silly me.

Still, glad to know it isn't REALLY this large!

DEEMAN223344 · Post by **DEEMAN223344** » Sun Jan 02, 2011 5:05 pm

I misread KB as MB.
IF it WAS 301,660 MB, I'd be scared.

boywhoflies · Post by **boywhoflies** » Sun Jan 02, 2011 5:08 pm

When I read that I saw(in my head, of course) you sitting in front of a computer, and the screen exploded and a bunch of 0s and 1s spewed out.

~xpr'd~ · Post by **~xpr'd~** » Sun Jan 02, 2011 5:14 pm

DEEMAN223344 wrote:OH! It's thirty MB, not GB. Silly me.

Well, glad to know it's fine.

Technos72 · Post by **Technos72** » Sun Jan 02, 2011 5:25 pm

DEEMAN223344 wrote:OH! It's thirty MB, not GB. Silly me.

Try figuring that out

Muzozavr · Post by **Muzozavr** » Sun Jan 02, 2011 5:32 pm

Technos72 wrote:
DEEMAN223344 wrote:OH! It's thirty MB, not GB. Silly me.

Try figuring that out

The image name already spells out the answer.

Marinus · Post by **Marinus** » Sun Jan 02, 2011 5:38 pm

The image name already spells out the answer.

I didn't even think of that, but I found it on a picture of the deaf-language alphabet.

tyteen4a03 · Post by **tyteen4a03** » Sun Jan 02, 2011 10:17 pm

(Notice: Mind your language, even it is not English.)
But anyways, 30MB makes much more sense to me.

maxnick · Post by **maxnick** » Mon Jan 03, 2011 11:46 pm

My reaction:

DEEMAN223344 wrote: THIRTY GIGABYTES, EVEN WHEN IN A .ZIP.

DEEMAN223344 wrote:OH! It's thirty MB, not GB. Silly me.

Oh.

StinkerSquad01 · Post by **StinkerSquad01** » Mon Jan 03, 2011 11:57 pm

Nice one.

When I saw it I was like, wait no, not possible!
When I read it was MB I was like, O ok.

Wow, that's a lousy compression.

Wow, that's a lousy compression.

Re: Wow, that's a lousy compression.