<FuSiyuan> | Hi :) | 2009-08-25 02:12:09 |
<Shelwien> | as to that size mismatch | 2009-08-25 02:28:40 |
| you could miss initialization of some variable(s) | 2009-08-25 02:28:54 |
| and also different results are easily possible if you use floats anywhere | 2009-08-25 02:29:25 |
| (well, not in statistics) | 2009-08-25 02:29:32 |
<FuSiyuan> | No floats | 2009-08-25 02:47:26 |
| you could miss initialization of some variable(s) ---- May be | 2009-08-25 02:47:42 |
<Shelwien> | there's one more possible reason - different structure member alignments | 2009-08-25 02:48:51 |
| and i don't remember any structures there | 2009-08-25 02:49:10 |
<FuSiyuan> | Yes.. No structure | 2009-08-25 02:52:10 |
<Shelwien> | he also could change something accidentally. | 2009-08-25 02:52:50 |
| your source doesn't compile without modifications ;) | 2009-08-25 02:53:13 |
<FuSiyuan> | :( | 2009-08-25 02:53:25 |
| That's No matter. I just paying more attention on next one | 2009-08-25 02:54:07 |
<Shelwien> | ;) | 2009-08-25 02:54:23 |
<FuSiyuan> | There is really no unversal compressor as you said | 2009-08-25 02:55:20 |
<Shelwien> | that's good actually ;) | 2009-08-25 02:55:41 |
| we'd have to search for another hobby otherwise ;) | 2009-08-25 02:56:14 |
<FuSiyuan> | I'm just start worrying about apply to graduate school next days | 2009-08-25 02:58:39 |
<Shelwien> | ? | 2009-08-25 02:59:06 |
| no more time for compression or what? ;) | 2009-08-25 02:59:55 |
<FuSiyuan> | Not too much. It costs too much attention | 2009-08-25 03:00:57 |
| Just got toefl score. | 2009-08-25 03:01:14 |
| In your country, toefl is fashion? | 2009-08-25 03:01:28 |
<Shelwien> | i'd not call it a fashion, but there're such tests, yeah | 2009-08-25 03:12:45 |
| i'm not from a english-speaking country obviously | 2009-08-25 03:13:11 |
| i'm russian, in fact, but live in Ukraine | 2009-08-25 03:13:32 |
<FuSiyuan> | Yes I mean if the test fashional among students | 2009-08-25 03:18:47 |
| I will measure what you told me while program next version. | 2009-08-25 03:20:15 |
<Shelwien> | i think its mandatory in some cases (like taking some courses in foreign countries) | 2009-08-25 03:20:41 |
| and students obviously discuss that | 2009-08-25 03:20:59 |
| but i'd not call it fashionable ;) | 2009-08-25 03:21:05 |
<FuSiyuan> | :) My inaccurate usage of this word | 2009-08-25 03:21:33 |
| I'm aiming at reasonable compression on reasonable speed with reasonable memory. : ) | 2009-08-25 03:22:44 |
<Shelwien> | what do you think about my "distribution metric"? | 2009-08-25 03:23:12 |
| i mean this: http://compressionratings.com/calc.cgi?file=rating_sum.full.html&id=10&cc=1&cd=10&cu=1&cw=10&bu=4096&bd=512&bt=1 | 2009-08-25 03:23:52 |
| the time of file compression + upload + 10 x download and decompression | 2009-08-25 03:24:50 |
<FuSiyuan> | Yes. I actually completely agree with it. I just didn't notice what is it before. | 2009-08-25 03:25:39 |
<Shelwien> | well, what kind of parameters do you think are good for it? | 2009-08-25 03:26:21 |
| like upload/download speeds and number of downloads? | 2009-08-25 03:26:38 |
<FuSiyuan> | Wiat. what does upload/download time for? | 2009-08-25 03:27:16 |
| Wait | 2009-08-25 03:27:19 |
<Shelwien> | you compress the file into size N | 2009-08-25 03:27:32 |
| then to upload that file with speed S | 2009-08-25 03:27:45 |
| which requires N/S time | 2009-08-25 03:27:56 |
| formula is like cTime + cSize/upSpeed + Downloaders*(cSize/downSpeed+dTime) | 2009-08-25 03:29:07 |
<FuSiyuan> | I think this is really a measurement for practical compression. | 2009-08-25 03:30:03 |
<Shelwien> | well, though actually Sami's version has separate coefficients for all 4 components | 2009-08-25 03:30:04 |
| yeah, its not the only one possible | 2009-08-25 03:30:36 |
<FuSiyuan> | I think the parameter isn't so important. Maybe 5x 10x or 20x. Bigger than 1 is ok. | 2009-08-25 03:31:10 |
<Shelwien> | it affects the decompression weight | 2009-08-25 03:31:35 |
<FuSiyuan> | Maybe average download times of compressed files | 2009-08-25 03:31:38 |
| Yes. I mean it's hard to decide the particular value. | 2009-08-25 03:32:04 |
<Shelwien> | if you make it 1000000 (which can be still practical for really popular files) | 2009-08-25 03:32:06 |
| then compression time won't matter at all | 2009-08-25 03:32:14 |
<FuSiyuan> | The compression rating itself is impossible to describe the performance completely. | 2009-08-25 03:33:29 |
| It's just a reference | 2009-08-25 03:33:47 |
<Shelwien> | a compressor is a practical tool | 2009-08-25 03:34:37 |
| so i think we have take some practical tasks for which compressors are used | 2009-08-25 03:34:57 |
| *to take | 2009-08-25 03:35:06 |
<FuSiyuan> | So what's why I think it's hard to decide what download weight exactly should be . | 2009-08-25 03:35:07 |
<Shelwien> | well, thats why its a calculator ;) | 2009-08-25 03:35:29 |
<FuSiyuan> | Obviously, symmetric compressor became so less competitive in this calculator | 2009-08-25 03:36:52 |
<Shelwien> | people actually feel like that too, so its more or less true | 2009-08-25 03:37:39 |
| i mean, that decompression speed is much more important | 2009-08-25 03:37:59 |
<FuSiyuan> | That's why I like LZ :) | 2009-08-25 03:38:10 |
<Shelwien> | some LZ part is certainly necessary because of redundant data, like Bulat's REP | 2009-08-25 03:38:51 |
| but for actual compression LZ is too bad | 2009-08-25 03:39:32 |
| its fairly bad for text | 2009-08-25 03:39:51 |
| and much worse for any other data types | 2009-08-25 03:40:01 |
<FuSiyuan> | Yesterday. I download XBW on the forum packed with bzip and extracted some files directly from the WinRAR several times. The decompression time became a obvious problem in such situation. | 2009-08-25 03:40:51 |
<Shelwien> | I guess, but in such case it doesn't really matter if its LZ or not ;) | 2009-08-25 03:41:35 |
<FuSiyuan> | However, as you see, on your calculator ranking, lz compressor ranks high. | 2009-08-25 03:41:45 |
| No. I don't think so . LZ decompressed this pack in less than 1s. | 2009-08-25 03:43:19 |
<Shelwien> | nanozip -co/O is BWT imho | 2009-08-25 03:43:53 |
| or LZ+BWT | 2009-08-25 03:44:23 |
<FuSiyuan> | Isn't it combination of LZ BWT? | 2009-08-25 03:44:30 |
<Shelwien> | ;) | 2009-08-25 03:44:39 |
<FuSiyuan> | Ilia so be fascinated in BWT | 2009-08-25 03:45:32 |
<Shelwien> | that's because its really simple | 2009-08-25 03:46:10 |
| and Ilia hates complex things ;) | 2009-08-25 03:46:28 |
<FuSiyuan> | However. I really don't know how it behaves in data except text or bmp | 2009-08-25 03:46:31 |
<Shelwien> | its rather bad on binary data | 2009-08-25 03:47:18 |
| unstable | 2009-08-25 03:47:20 |
<FuSiyuan> | Yes. | 2009-08-25 03:49:33 |
<Shelwien> | and also it can't handle the long matches | 2009-08-25 03:50:15 |
<FuSiyuan> | unstable. So bcm ranked not so high in compression ratings. | 2009-08-25 03:50:20 |
| I have ever sent to Ilia what if preprocessing like rep combined with bmc | 2009-08-25 03:51:36 |
| bcm | 2009-08-25 03:51:37 |
<Shelwien> | he won't do that. he even used libdivsufsort for BWT ;) | 2009-08-25 03:53:50 |
<FuSiyuan> | libdivsufsort ? | 2009-08-25 03:54:31 |
<Shelwien> | a BWT library | 2009-08-25 03:54:40 |
<FuSiyuan> | Oh.. | 2009-08-25 03:54:47 |
<Shelwien> | well, postcoder is important too | 2009-08-25 03:55:26 |
<FuSiyuan> | What's postcoder ? | 2009-08-25 03:55:46 |
<Shelwien> | BWT compressors consist of two unrelated parts | 2009-08-25 03:56:11 |
<FuSiyuan> | Yes sorting and coding | 2009-08-25 03:56:23 |
| right? | 2009-08-25 03:56:27 |
<Shelwien> | BWT itself (string sorting) and coding | 2009-08-25 03:56:30 |
| yeah | 2009-08-25 03:56:32 |
| coding in BWT compressor is sometimes called "postcoding" | 2009-08-25 03:56:48 |
| because its done after transformation ;) | 2009-08-25 03:56:58 |
<FuSiyuan> | Oh.. I see | 2009-08-25 03:57:21 |
| Yes. I found him very devoted into the cm part of BCM. | 2009-08-25 03:57:52 |
<Shelwien> | well, its what he actually written there | 2009-08-25 03:58:23 |
<FuSiyuan> | B CM.... | 2009-08-25 03:58:29 |
<Shelwien> | and also not only there | 2009-08-25 03:58:36 |
| i think he used more or less the same approach in most of his recent compressors | 2009-08-25 03:59:10 |
| i mean, LZ requires literal coding etc too | 2009-08-25 03:59:40 |
| and it can be done by a similar CM as the one used for BWT postcoding | 2009-08-25 04:00:17 |
<FuSiyuan> | I've only carefully read his BALZ | 2009-08-25 04:00:21 |
| Just order1. No much tricks. | 2009-08-25 04:01:11 |
<Shelwien> | sure. so he'd progressed since then ;) | 2009-08-25 04:01:55 |
<FuSiyuan> | I just no higher than this level. :( | 2009-08-25 04:03:06 |
<Shelwien> | there's a lot of sources to learn from, so i don't know what's the problem | 2009-08-25 04:03:45 |
| as i said, if i were at your place, i'd just take the LZMA source | 2009-08-25 04:04:12 |
<FuSiyuan> | Yes. Just spend time. | 2009-08-25 04:04:22 |
<Shelwien> | and tried to clean it to make more easy to understand and use (for myself) | 2009-08-25 04:04:37 |
<FuSiyuan> | you mean. To make some improvements based on it? | 2009-08-25 04:05:26 |
<Shelwien> | eventually, yes, that's what I'd do | 2009-08-25 04:05:39 |
*** pinc has left the channel | 2009-08-25 04:05:51 |
| there's no perfect programs anyway | 2009-08-25 04:05:54 |
| also popular codecs, like LZMA, have to care too much about format compatibility | 2009-08-25 04:06:49 |
| and there're usually too much of redundant code | 2009-08-25 04:07:38 |
| like, ppmd source is originally ~60k | 2009-08-25 04:08:03 |
| and ppmd_sh is ~30k | 2009-08-25 04:08:07 |
| and like half of that still can be removed | 2009-08-25 04:08:27 |
<FuSiyuan> | I think I should learn some experiences while reading the source. However I just like to work all by myself from the lowest layer. | 2009-08-25 04:08:58 |
<Shelwien> | well, as you can see i do both | 2009-08-25 04:09:27 |
<FuSiyuan> | You are a much more advanced programmer. ;) | 2009-08-25 04:09:50 |
<Shelwien> | its never late to learn something new ;) | 2009-08-25 04:09:54 |
<FuSiyuan> | Right | 2009-08-25 04:10:01 |
<Shelwien> | like, we found a few interesting tricks in ccm these days | 2009-08-25 04:10:17 |
<FuSiyuan> | How long it is since you get into this field? | 2009-08-25 04:11:18 |
<Shelwien> | around 10 years i guess | 2009-08-25 04:12:46 |
<FuSiyuan> | ... | 2009-08-25 04:12:51 |
<Shelwien> | and nearly 20 of overall programming experience ;) | 2009-08-25 04:13:48 |
<FuSiyuan> | ;) | 2009-08-25 04:14:05 |
<Shelwien> | the "experience" from pre-x86 computers is not of much use though ;) | 2009-08-25 04:14:49 |
<FuSiyuan> | So much should to learn for me. | 2009-08-25 04:14:49 |
| Just 8month and 2 years. | 2009-08-25 04:15:12 |
<Shelwien> | well, i'm not sure if it means much really | 2009-08-25 04:15:49 |
<FuSiyuan> | :) | 2009-08-25 04:16:06 |
<Shelwien> | like, i used to program in assembly most of that time | 2009-08-25 04:16:07 |
| and i can't really do that anymore | 2009-08-25 04:16:18 |
| because there're things that compilers would certainly do better than me | 2009-08-25 04:16:44 |
| or, more like, i don't have time to do that manually | 2009-08-25 04:17:10 |
| and nobody makes any good tools for assembly programming anymore | 2009-08-25 04:17:27 |
| i mean | 2009-08-25 04:18:10 |
| so guy fresh from a C# course can have more jobs than I'd be able to find | 2009-08-25 04:18:38 |
<FuSiyuan> | I think it's really necessary to have the assembly architecture knowledge. I lack of these and be learning these. | 2009-08-25 04:19:39 |
<Shelwien> | well, its certainly necessary for C/C++ programming | 2009-08-25 04:20:12 |
| in that sense, I'd really recommend reading that - http://www.agner.org/optimize/ | 2009-08-25 04:20:35 |
| but C++ is long out of fashion | 2009-08-25 04:21:14 |
| and java/C# are rather unrelated to a real system architecture | 2009-08-25 04:22:07 |
| or python/ruby/whatever else is popular | 2009-08-25 04:22:27 |
| basically, only your ability to solve the problems matters | 2009-08-25 04:23:17 |
| and there can be very different tools and approaches for that | 2009-08-25 04:23:29 |
<FuSiyuan> | Just a phone call. Glad talking these with you, however I gonna out with frends. | 2009-08-25 04:26:10 |
<Shelwien> | good luck ;) | 2009-08-25 04:26:37 |
*** FuSiyuan has left the channel | 2009-08-25 04:27:45 |
| !next | 2009-08-25 04:27:49 |