<FuSiyuan> Hi :)2009-08-25 02:12:09
<Shelwien> as to that size mismatch2009-08-25 02:28:40
 you could miss initialization of some variable(s)2009-08-25 02:28:54
 and also different results are easily possible if you use floats anywhere2009-08-25 02:29:25
 (well, not in statistics)2009-08-25 02:29:32
<FuSiyuan> No floats2009-08-25 02:47:26
 you could miss initialization of some variable(s) ---- May be2009-08-25 02:47:42
<Shelwien> there's one more possible reason - different structure member alignments2009-08-25 02:48:51
 and i don't remember any structures there2009-08-25 02:49:10
<FuSiyuan>  Yes.. No structure2009-08-25 02:52:10
<Shelwien> he also could change something accidentally.2009-08-25 02:52:50
 your source doesn't compile without modifications ;)2009-08-25 02:53:13
<FuSiyuan> :(2009-08-25 02:53:25
 That's No matter. I just paying more attention on next one2009-08-25 02:54:07
<Shelwien> ;)2009-08-25 02:54:23
<FuSiyuan>  There is really no unversal compressor as you said2009-08-25 02:55:20
<Shelwien> that's good actually ;)2009-08-25 02:55:41
 we'd have to search for another hobby otherwise ;)2009-08-25 02:56:14
<FuSiyuan> I'm just start worrying about apply to graduate school next days2009-08-25 02:58:39
<Shelwien> ?2009-08-25 02:59:06
 no more time for compression or what? ;)2009-08-25 02:59:55
<FuSiyuan> Not too much. It costs too much attention2009-08-25 03:00:57
 Just got toefl score.2009-08-25 03:01:14
  In your country, toefl is fashion?2009-08-25 03:01:28
<Shelwien> i'd not call it a fashion, but there're such tests, yeah2009-08-25 03:12:45
 i'm not from a english-speaking country obviously2009-08-25 03:13:11
 i'm russian, in fact, but live in Ukraine2009-08-25 03:13:32
<FuSiyuan> Yes I mean if the test fashional among students 2009-08-25 03:18:47
  I will measure what you told me while program next version.2009-08-25 03:20:15
<Shelwien> i think its mandatory in some cases (like taking some courses in foreign countries)2009-08-25 03:20:41
 and students obviously discuss that2009-08-25 03:20:59
 but i'd not call it fashionable ;)2009-08-25 03:21:05
<FuSiyuan>  :) My inaccurate usage of this word2009-08-25 03:21:33
 I'm aiming at reasonable compression on reasonable speed with reasonable memory. : ) 2009-08-25 03:22:44
<Shelwien> what do you think about my "distribution metric"?2009-08-25 03:23:12
 i mean this: http://compressionratings.com/calc.cgi?file=rating_sum.full.html&id=10&cc=1&cd=10&cu=1&cw=10&bu=4096&bd=512&bt=12009-08-25 03:23:52
 the time of file compression + upload + 10 x download and decompression2009-08-25 03:24:50
<FuSiyuan> Yes. I actually completely agree with it. I just didn't notice what is it before.2009-08-25 03:25:39
<Shelwien> well, what kind of parameters do you think are good for it?2009-08-25 03:26:21
 like upload/download speeds and number of downloads?2009-08-25 03:26:38
<FuSiyuan> Wiat. what does upload/download time for?2009-08-25 03:27:16
 Wait2009-08-25 03:27:19
<Shelwien> you compress the file into size N2009-08-25 03:27:32
 then to upload that file with speed S2009-08-25 03:27:45
 which requires N/S time2009-08-25 03:27:56
 formula is like cTime + cSize/upSpeed + Downloaders*(cSize/downSpeed+dTime)2009-08-25 03:29:07
<FuSiyuan>  I think this is really a measurement for practical compression.2009-08-25 03:30:03
<Shelwien> well, though actually Sami's version has separate coefficients for all 4 components2009-08-25 03:30:04
 yeah, its not the only one possible2009-08-25 03:30:36
<FuSiyuan>  I think the parameter isn't so important. Maybe 5x 10x or 20x. Bigger than 1 is ok.2009-08-25 03:31:10
<Shelwien> it affects the decompression weight2009-08-25 03:31:35
<FuSiyuan>  Maybe average download times of compressed files2009-08-25 03:31:38
 Yes. I mean it's hard to decide the particular value.2009-08-25 03:32:04
<Shelwien> if you make it 1000000 (which can be still practical for really popular files)2009-08-25 03:32:06
 then compression time won't matter at all2009-08-25 03:32:14
<FuSiyuan> The compression rating itself is impossible to describe the performance completely.2009-08-25 03:33:29
 It's just a reference2009-08-25 03:33:47
<Shelwien> a compressor is a practical tool2009-08-25 03:34:37
 so i think we have take some practical tasks for which compressors are used2009-08-25 03:34:57
 *to take2009-08-25 03:35:06
<FuSiyuan> So what's why I think it's hard to decide what download weight exactly should be . 2009-08-25 03:35:07
<Shelwien> well, thats why its a calculator ;)2009-08-25 03:35:29
<FuSiyuan> Obviously, symmetric compressor became so less competitive in this calculator2009-08-25 03:36:52
<Shelwien> people actually feel like that too, so its more or less true2009-08-25 03:37:39
 i mean, that decompression speed is much more important2009-08-25 03:37:59
<FuSiyuan> That's why I like LZ :)2009-08-25 03:38:10
<Shelwien> some LZ part is certainly necessary because of redundant data, like Bulat's REP2009-08-25 03:38:51
 but for actual compression LZ is too bad2009-08-25 03:39:32
 its fairly bad for text2009-08-25 03:39:51
 and much worse for any other data types2009-08-25 03:40:01
<FuSiyuan> Yesterday. I download XBW on the forum packed with bzip and extracted some files directly from the WinRAR several times. The decompression time became a obvious problem in such situation.2009-08-25 03:40:51
<Shelwien> I guess, but in such case it doesn't really matter if its LZ or not ;)2009-08-25 03:41:35
<FuSiyuan> However, as you see, on your calculator ranking, lz compressor ranks high.2009-08-25 03:41:45
  No. I don't think so . LZ decompressed this pack in less than 1s.2009-08-25 03:43:19
<Shelwien> nanozip -co/O is BWT imho2009-08-25 03:43:53
 or LZ+BWT2009-08-25 03:44:23
<FuSiyuan> Isn't it combination of LZ BWT?2009-08-25 03:44:30
<Shelwien> ;)2009-08-25 03:44:39
<FuSiyuan> Ilia so be fascinated in BWT2009-08-25 03:45:32
<Shelwien> that's because its really simple2009-08-25 03:46:10
 and Ilia hates complex things ;)2009-08-25 03:46:28
<FuSiyuan> However. I really don't know how it behaves in data except text or bmp2009-08-25 03:46:31
<Shelwien> its rather bad on binary data2009-08-25 03:47:18
 unstable2009-08-25 03:47:20
<FuSiyuan> Yes.2009-08-25 03:49:33
<Shelwien> and also it can't handle the long matches2009-08-25 03:50:15
<FuSiyuan>  unstable. So bcm ranked not so high in compression ratings.2009-08-25 03:50:20
  I have ever sent to Ilia what if preprocessing like rep combined with bmc2009-08-25 03:51:36
 bcm2009-08-25 03:51:37
<Shelwien> he won't do that. he even used libdivsufsort for BWT ;)2009-08-25 03:53:50
<FuSiyuan> libdivsufsort ?2009-08-25 03:54:31
<Shelwien> a BWT library2009-08-25 03:54:40
<FuSiyuan>  Oh..2009-08-25 03:54:47
<Shelwien> well, postcoder is important too2009-08-25 03:55:26
<FuSiyuan> What's postcoder ?2009-08-25 03:55:46
<Shelwien> BWT compressors consist of two unrelated parts2009-08-25 03:56:11
<FuSiyuan> Yes sorting and coding2009-08-25 03:56:23
 right?2009-08-25 03:56:27
<Shelwien> BWT itself (string sorting) and coding2009-08-25 03:56:30
 yeah2009-08-25 03:56:32
 coding in BWT compressor is sometimes called "postcoding"2009-08-25 03:56:48
 because its done after transformation ;)2009-08-25 03:56:58
<FuSiyuan> Oh.. I see2009-08-25 03:57:21
 Yes. I found him very devoted into the cm part of BCM.2009-08-25 03:57:52
<Shelwien> well, its what he actually written there2009-08-25 03:58:23
<FuSiyuan> B CM....2009-08-25 03:58:29
<Shelwien> and also not only there2009-08-25 03:58:36
 i think he used more or less the same approach in most of his recent compressors2009-08-25 03:59:10
 i mean, LZ requires literal coding etc too2009-08-25 03:59:40
 and it can be done by a similar CM as the one used for BWT postcoding2009-08-25 04:00:17
<FuSiyuan> I've only carefully read his BALZ2009-08-25 04:00:21
 Just order1. No much tricks.2009-08-25 04:01:11
<Shelwien> sure. so he'd progressed since then ;)2009-08-25 04:01:55
<FuSiyuan> I just no higher than this level. :(2009-08-25 04:03:06
<Shelwien> there's a lot of sources to learn from, so i don't know what's the problem2009-08-25 04:03:45
 as i said, if i were at your place, i'd just take the LZMA source2009-08-25 04:04:12
<FuSiyuan> Yes. Just spend time.2009-08-25 04:04:22
<Shelwien> and tried to clean it to make more easy to understand and use (for myself)2009-08-25 04:04:37
<FuSiyuan> you mean. To make some improvements based on it?2009-08-25 04:05:26
<Shelwien> eventually, yes, that's what I'd do2009-08-25 04:05:39
*** pinc has left the channel2009-08-25 04:05:51
 there's no perfect programs anyway2009-08-25 04:05:54
 also popular codecs, like LZMA, have to care too much about format compatibility2009-08-25 04:06:49
 and there're usually too much of redundant code2009-08-25 04:07:38
 like, ppmd source is originally ~60k2009-08-25 04:08:03
 and ppmd_sh is ~30k2009-08-25 04:08:07
 and like half of that still can be removed2009-08-25 04:08:27
<FuSiyuan> I think I should learn some experiences while reading the source. However I just like to work all by myself from the lowest layer.2009-08-25 04:08:58
<Shelwien> well, as you can see i do both2009-08-25 04:09:27
<FuSiyuan> You are a much more advanced programmer. ;)2009-08-25 04:09:50
<Shelwien> its never late to learn something new ;)2009-08-25 04:09:54
<FuSiyuan> Right2009-08-25 04:10:01
<Shelwien> like, we found a few interesting tricks in ccm these days2009-08-25 04:10:17
<FuSiyuan> How long it is since you get into this field?2009-08-25 04:11:18
<Shelwien> around 10 years i guess2009-08-25 04:12:46
<FuSiyuan> ...2009-08-25 04:12:51
<Shelwien> and nearly 20 of overall programming experience ;)2009-08-25 04:13:48
<FuSiyuan>  ;)2009-08-25 04:14:05
<Shelwien> the "experience" from pre-x86 computers is not of much use though ;)2009-08-25 04:14:49
<FuSiyuan> So much should to learn for me.2009-08-25 04:14:49
 Just 8month and 2 years.2009-08-25 04:15:12
<Shelwien> well, i'm not sure if it means much really2009-08-25 04:15:49
<FuSiyuan> :)2009-08-25 04:16:06
<Shelwien> like, i used to program in assembly most of that time2009-08-25 04:16:07
 and i can't really do that anymore2009-08-25 04:16:18
 because there're things that compilers would certainly do better than me2009-08-25 04:16:44
 or, more like, i don't have time to do that manually2009-08-25 04:17:10
 and nobody makes any good tools for assembly programming anymore2009-08-25 04:17:27
 i mean2009-08-25 04:18:10
 so guy fresh from a C# course can have more jobs than I'd be able to find2009-08-25 04:18:38
<FuSiyuan> I think it's really necessary to have the assembly architecture knowledge. I lack of these and be learning these. 2009-08-25 04:19:39
<Shelwien> well, its certainly necessary for C/C++ programming2009-08-25 04:20:12
 in that sense, I'd really recommend reading that - http://www.agner.org/optimize/2009-08-25 04:20:35
 but C++ is long out of fashion2009-08-25 04:21:14
 and java/C# are rather unrelated to a real system architecture2009-08-25 04:22:07
 or python/ruby/whatever else is popular2009-08-25 04:22:27
 basically, only your ability to solve the problems matters2009-08-25 04:23:17
 and there can be very different tools and approaches for that2009-08-25 04:23:29
<FuSiyuan> Just a phone call. Glad talking these with you, however I gonna out with frends.2009-08-25 04:26:10
<Shelwien> good luck ;)2009-08-25 04:26:37
*** FuSiyuan has left the channel2009-08-25 04:27:45
 !next2009-08-25 04:27:49