*** Shelwien has left the channel2009-11-19 02:11:31
*** Shelwien has joined the channel2009-11-19 02:11:35
*** pinc has joined the channel2009-11-19 07:53:13
*** pinc has left the channel2009-11-19 08:57:07
*** pinc has joined the channel2009-11-19 09:26:30
*** toffer has joined the channel2009-11-19 10:26:19
*** pinc|mirror has joined the channel2009-11-19 12:41:41
*** pinc has left the channel2009-11-19 12:41:41
*** pinc|mirror has left the channel2009-11-19 12:46:09
*** pinc has joined the channel2009-11-19 12:48:37
*** pinc|mirror has joined the channel2009-11-19 12:57:11
*** pinc has left the channel2009-11-19 12:57:11
*** moisesmcardona has joined the channel2009-11-19 13:11:37
*** moisesmcardona has left the channel2009-11-19 13:13:43
*** pinc|mirror has left the channel2009-11-19 17:04:24
*** Shelwien has left the channel2009-11-19 17:51:02
*** pinc has joined the channel2009-11-19 17:59:52
*** toffer has left the channel2009-11-19 19:55:40
*** pinc has left the channel2009-11-19 20:47:35
*** Shelwien has joined the channel2009-11-19 23:47:24
*** Shelwien has left the channel2009-11-20 06:30:20
*** Shelwien has joined the channel2009-11-20 06:33:33
*** Shelwien has left the channel2009-11-20 06:42:50
*** Shelwien has joined the channel2009-11-20 07:34:38
*** chornobyl has joined the channel2009-11-20 10:07:47
<Shelwien> 2009-11-20 10:26:16 weintainypeIo...> ...please where can I buy a unicorn?2009-11-20 10:34:03
 this spam is funny2009-11-20 10:34:14
 http://ctxmodel.net/rem.pl?82009-11-20 10:34:23
<chornobyl> lot of it2009-11-20 10:34:43
* Shelwien is installing MS SQL Server 2008. To investigate its backup compression2009-11-20 10:35:45
*** toffer has joined the channel2009-11-20 10:54:10
<toffer> hi2009-11-20 10:57:34
<Shelwien> hi2009-11-20 10:57:42
<chornobyl> hello2009-11-20 10:57:47
<toffer> i finished the slides for my presentation this morning at 4 - feeling a bit tired now2009-11-20 10:59:37
 any news, meanwhile?2009-11-20 10:59:52
<Shelwien> not much. i'm writing a new filesystem scanner2009-11-20 11:00:43
 and all this stuff with unicode, codepages, and windows API2009-11-20 11:01:14
<toffer> you've already written one, don't you?!2009-11-20 11:01:15
<Shelwien> goes quite slow2009-11-20 11:01:20
 well, i'm writing a new one now ;)2009-11-20 11:01:34
 in hope that it would be even better ;)2009-11-20 11:01:48
 also this one is for my backup engine2009-11-20 11:02:21
<toffer> good luck than2009-11-20 11:02:41
 gonna have some breakfast now2009-11-20 11:02:46
 btw did i already mention that my time series predictor outpreformed the fraunhofer one?2009-11-20 11:03:03
<Shelwien> yeah2009-11-20 11:03:14
 that's kinda expected though ;)2009-11-20 11:03:23
<toffer> self-praise sucks :D2009-11-20 11:03:47
 and i think when i've got my spare time back again i'll modify m1 to get into the ltcb top ten :D2009-11-20 11:04:59
 somehow lpaq shows better performance when dropping models (!)2009-11-20 11:05:21
<Shelwien> if mean if you remove models from lpaq?2009-11-20 11:05:55
<toffer> yes2009-11-20 11:06:00
 order 3 is useless for example2009-11-20 11:06:05
 almost no change2009-11-20 11:06:07
 to test some of my ideas2009-11-20 11:06:26
 i modified it to use orders 248, match and word2009-11-20 11:06:58
 it produces nearly the same compression2009-11-20 11:07:08
 on the ltcb2009-11-20 11:07:11
<Shelwien> well, its special2009-11-20 11:07:27
<toffer> thus if you'd add a dictionary to that2009-11-20 11:07:27
 you'd get a top 10 entry2009-11-20 11:07:36
 i know2009-11-20 11:07:42
 it's not universal2009-11-20 11:07:48
 still these chained sses are not useful - they mostly eat up cpu time2009-11-20 11:08:04
<Shelwien> yeah, Matt seems to still have troubles with SSE handling2009-11-20 11:08:59
<toffer> i mean just 4-5 specialized models are enough to produce very good compression2009-11-20 11:09:15
 and one or two of these can be implemenetd via direct lookup2009-11-20 11:10:56
 guess i'll add a modification to have one or two "low order models" for speed2009-11-20 11:11:58
*** Shelwien has left the channel2009-11-20 11:13:49
*** Shelwien has joined the channel2009-11-20 11:14:02
 and i found that sse for mixing is faster than logistic mixing - well if it isn't vectorized2009-11-20 11:14:44
<Shelwien> yeah2009-11-20 11:15:18
<toffer> since prediction is just a lookup2009-11-20 11:15:28
<Shelwien> i think Matt didn't ever use SSE2+ too2009-11-20 11:15:31
 though i guess it should be possible to add some logistic extrapolation to SSE2009-11-20 11:16:02
<toffer> do you mean to a sse counter or via using input probs. explicitly_2009-11-20 11:17:03
 and something else - what kind of probability estimation is shkarin use? i remember some weird stuff "substract mean" or something like that in a comment?2009-11-20 11:19:33
<Shelwien> well, there's sse probability estimation2009-11-20 11:19:42
 and even if you apply SSE in logistic space2009-11-20 11:19:59
 its still possible to add some multiplier after SSE2009-11-20 11:20:17
<toffer> multiplier in the sense of rescaling?2009-11-20 11:20:56
<Shelwien> and Shkarin mostly uses integer frequencies everywhere2009-11-20 11:21:04
 but as to means etc - why not2009-11-20 11:21:20
 i mean, its a kind of linear extrapolation etc2009-11-20 11:22:17
 ...and multiplier like in sq(C*st(p))2009-11-20 11:22:49
<toffer> not that important. it just came to my mind these days and i didn't know what that would be intended for2009-11-20 11:22:51
 yeah so rescaling2009-11-20 11:22:58
<Shelwien> i'd say "rescaling" is something different ;)2009-11-20 11:23:20
<toffer> C>1 ?2009-11-20 11:23:22
 well it is the same2009-11-20 11:23:29
<Shelwien> probably, but not necessarily2009-11-20 11:23:40
 i think i had it tuned to <1 somewhere too2009-11-20 11:23:55
<toffer> but the steps will be larger near 0.5 and smaller near 0/12009-11-20 11:23:58
 step = delta sq(..) vs. delta C2009-11-20 11:24:24
<Shelwien> dunno, i have 15 bit precision in logistic domain too2009-11-20 11:24:31
 and C* is done in fixed point obviously2009-11-20 11:24:55
 C*s>>15 or something2009-11-20 11:25:10
<toffer> well that's clear2009-11-20 11:25:24
 i meant comparing the effect in probability domain2009-11-20 11:25:35
<Shelwien> should be smooth enough, dunno2009-11-20 11:26:12
 and anyway, its probably possible to invent a better extrapolation for SSE2009-11-20 11:28:43
 with likelihoods etc2009-11-20 11:28:52
 there're two mixed points, but the logic is somewhat different from mixer2009-11-20 11:30:07
<toffer> two mixed points?2009-11-20 11:31:57
<Shelwien> ah, i forgot2009-11-20 11:32:07
 i'm talking about interpolated 1D SSE obviously2009-11-20 11:32:23
<toffer> sq(C*st(p)) seems to be 1 dimensional :) 2009-11-20 11:34:13
 still i don't know what "two mixed points" should be2009-11-20 11:34:33
<Shelwien> well, SSE can be interpolated2009-11-20 11:35:28
<toffer> yes2009-11-20 11:35:53
 but why'd you want to interpolate "sq(C*st(p))"2009-11-20 11:36:13
<Shelwien> so there'd be some sse[q]*r+sse[q+1]*(1-r)2009-11-20 11:36:31
<toffer> ok 2009-11-20 11:36:36
 ah2009-11-20 11:36:42
 now i know what you actually mean2009-11-20 11:36:48
<Shelwien> i meant two unrelated things actually2009-11-20 11:37:01
<toffer> erm?2009-11-20 11:37:23
<Shelwien> 1. that sq(C*SSE(p)) can be a used after SSE as a posterior extrapolation2009-11-20 11:38:04
<toffer> afaiu you want to replace the predictions in a "bin"-based sse (prob -> bin, weight) with refined predictions "sse[q]*r+sse[q+1]*(1-r)"2009-11-20 11:38:35
<Shelwien> 2. that interpolated SSE is kinda like mixing, but its logic is completely different, so if we'd think about it, the underlying model etc might be different too2009-11-20 11:38:46
 so different likelihoods etc2009-11-20 11:39:01
 and thus different mixing in likelihood domain2009-11-20 11:39:19
<toffer> wouldn't it be best to use any of these mappings after sse2009-11-20 11:40:22
 mh2009-11-20 11:40:43
 well these mappings are satic?2009-11-20 11:40:48
 sq(C*SSE(p)), C is constant during processing2009-11-20 11:41:04
<Shelwien> yeah2009-11-20 11:41:45
<toffer> at least i now know what you're meaning with sse and mixing have in common2009-11-20 11:41:56
<Shelwien> well, i'm actually trying to say2009-11-20 11:42:31
 that interpolated SSE in logistic domain2009-11-20 11:42:42
 might have to be completely different from paq mixer2009-11-20 11:42:51
 as it can't be nicely approximated by a switching model like mixer2009-11-20 11:43:34
<toffer> sure? selecting a single bin works for me2009-11-20 11:44:04
 i mean the nearest2009-11-20 11:44:08
<Shelwien> yeah, whatever2009-11-20 11:44:16
<toffer> r<0.5 ? 0 : 12009-11-20 11:44:19
 r<0.5 ? left : right2009-11-20 11:44:29
 or something like that2009-11-20 11:44:36
<Shelwien> i know, been there2009-11-20 11:44:40
<toffer> but i think i understood what you said2009-11-20 11:44:49
<Shelwien> i've only used linear interpolated SSE (and SSE2) until now2009-11-20 11:45:28
<toffer> it works even quite well for 2d sse2009-11-20 11:45:55
<Shelwien> but i also did some experiments with nonlinear mapping for the probability2009-11-20 11:45:58
<toffer> i'm using it all the time2009-11-20 11:46:00
<Shelwien> and it helped a little, even though that was some ad hoc nonlinearity, not anything like stretch2009-11-20 11:46:55
 although, in fact2009-11-20 11:48:07
 my interpolated SSE works good enough as is too, somehow2009-11-20 11:48:22
 ash shows like 202k on book12009-11-20 11:48:46
 and there're no word models or anything like that2009-11-20 11:49:00
 ...2009-11-20 11:50:50
 and again, what i mean is2009-11-20 11:50:59
 that my explanation for paq mixer2009-11-20 11:51:07
 was based on likelihoods for an approximate model2009-11-20 11:51:36
 looking like two static probability estimation, switched with a static probability2009-11-20 11:52:02
 *estimations2009-11-20 11:52:15
 and that obviously doesn't apply to SSE2009-11-20 11:53:04
<toffer> i think interpreting sse like that might not be right. since the "mixing" is just linear interpolation. there might be other solutions which work better2009-11-20 11:54:16
<Shelwien> like what?2009-11-20 11:54:41
<toffer> sse[q]*r+sse[q+1]*(1-r)2009-11-20 11:55:00
<Shelwien> you quantize the input probability2009-11-20 11:55:04
 and that leaves a remainder2009-11-20 11:55:21
<toffer> yeah2009-11-20 11:55:25
 i know2009-11-20 11:55:26
<Shelwien> but, the same likelihood approach apply here too2009-11-20 11:55:41
 estimating likelihoods for n0+1 case and for n1+1 case2009-11-20 11:55:58
 and afaik2009-11-20 11:57:00
 that would give a significantly different function from logistic mixing2009-11-20 11:57:15
 though obviously with log/exp too2009-11-20 11:57:30
<toffer> sorry have to leave now for gym & travelling home 2009-11-20 12:04:25
<Shelwien> ok, i'd sleep anyway2009-11-20 12:04:44
<toffer> so "good night" - or "good day" :D2009-11-20 12:05:58
<Shelwien> ;)2009-11-20 12:06:10
*** toffer has left the channel2009-11-20 12:07:40
*** chornobyl has left the channel2009-11-20 12:57:13
*** zapper has joined the channel2009-11-20 16:36:40
* zapper slaps complogger around a bit with a large fishbot2009-11-20 16:37:17
* zapper slaps Shelwien around a bit with a large fishbot2009-11-20 16:38:05
* zapper slaps asmodean around a bit with a large fishbot2009-11-20 16:38:20
*** zapper has left the channel2009-11-20 16:38:30
*** Shelwien has left the channel2009-11-21 11:13:09
*** Guest9968193 has joined the channel2009-11-21 11:13:13
*** pinc has joined the channel2009-11-21 17:28:27
*** red has joined the channel2009-11-21 18:20:20
<red> hey all2009-11-21 18:20:29
<Shelwien> hi2009-11-21 18:20:57
<red> hi2009-11-21 18:21:43
 just read your "Directory scanning in windows" post ..nice2009-11-21 18:22:37
<Shelwien> well, i'd say its quite troublesome2009-11-21 18:23:34
<red> informative..nonetheless 2009-11-21 18:24:12
<Shelwien> did you see the benchmarks? ;)2009-11-21 18:24:26
*** kj has joined the channel2009-11-21 18:26:16
*** kj has left the channel2009-11-21 18:26:21
<red> what benchmarks ??2009-11-21 18:27:17
*** pinc has left the channel2009-11-21 18:27:18
<Shelwien> there's some in http://shelwien.googlepages.com/sami_template_v2.rar2009-11-21 18:27:56
 ah, also this: http://compressionratings.com/s_scan.html2009-11-21 18:29:57
<red> this is good2009-11-21 18:31:06
<Shelwien> hope so2009-11-21 18:31:39
 unfortunately only my implementation supports unicode2009-11-21 18:31:58
 ...also, there was a java version too2009-11-21 18:32:52
 only twice slower than Sami's STL ;)2009-11-21 18:33:02
<red> i've always seen posts by you that have a code snippet or benchmarks ...that really helps thank you2009-11-21 18:33:34
<Shelwien> well, i'm kinda trying to start some discussion with these2009-11-21 18:35:11
 no luck most of the time, though ;)2009-11-21 18:35:18
 for example, there's also a quirk like this (in directory scanning):2009-11-21 18:37:17
 if we use "." or ".." as a base path2009-11-21 18:37:33
*** red_ has joined the channel2009-11-21 18:37:44
<red_> back2009-11-21 18:38:13
<Shelwien> the actual directory name would be substituted instead of "." part2009-11-21 18:38:34
 check http://ps16893.dreamhost.com/cgi-system/chantail.cgi?32 if you missed anything2009-11-21 18:38:47
 so, if we make an archiver using my implementation2009-11-21 18:39:24
*** red has left the channel2009-11-21 18:39:41
 it would probably add the current directory with a name into archive, if we'd start it using "."2009-11-21 18:40:02
 while eg. rar doesn't do that - there it works the same as .\*2009-11-21 18:40:54
 so which is better? ;)2009-11-21 18:41:19
<red> hmmm....2009-11-21 18:41:45
 i really dont know?2009-11-21 18:44:23
<Shelwien> that's the problem ;)2009-11-21 18:44:43
 because rar's behavior is much harder to implement actually2009-11-21 18:45:06
 because both FindFirst and GetFullPathName would convert that to a real directory name2009-11-21 18:45:53
 so we have to modify the base path from "." to ".\*" before converting it into canonical form2009-11-21 18:46:34
 but then, there can be stuff like "C:."2009-11-21 18:47:06
 or something like "\\?\C:\1\." even2009-11-21 18:47:21
 basically, that's like rewriting that GetFullPathName2009-11-21 18:47:53
 ...i really wonder why system APIs always have to be crazy...2009-11-21 18:48:26
<red> wouldnt something like regular expression help with this sort of this2009-11-21 18:49:02
<Shelwien> don't forget about unicode ;)2009-11-21 18:49:19
 well, i guess it would, after conversion to utf8 or something2009-11-21 18:49:58
 and kinda that's why people don't write anything in plain C/C++ anymore2009-11-21 18:50:42
 but then we get bloated programs2009-11-21 18:51:05
<red>  a temporary thing /a method that invokes the "regular ex" when some troublesome "pathname" comes up2009-11-21 18:52:18
 else it goes on to process like it does currently2009-11-21 18:53:07
<Shelwien> yeah, but as i said, there're no regexps in plain C++ - it comes as a separate library2009-11-21 18:54:03
 and there's no build-in string type actually2009-11-21 18:54:16
 *built-in2009-11-21 18:54:25
<red> i understand2009-11-21 18:55:08
<Shelwien> but my scanner in C++ can be like 4k executable (depending on compiler options)2009-11-21 18:55:09
 and equivalent implementation in perl, for example2009-11-21 18:55:42
 would require like 3 dlls to distribute with it2009-11-21 18:55:55
<red> but what good a 4k scanner it it cant work ?2009-11-21 18:56:11
 *if2009-11-21 18:56:18
<Shelwien> err, but it can2009-11-21 18:56:27
 even though it might behave a little different from what other programs do2009-11-21 18:56:57
 but on other hand2009-11-21 18:57:06
 most of these "other programs" might have other - and more troublesome - problems2009-11-21 18:57:25
 like not understanding national filenames etc2009-11-21 18:58:17
 and there's a question of time/speed too2009-11-21 18:58:59
<red> yes2009-11-21 18:59:22
<Shelwien> like, my scan5 takes 4s to scan some directory2009-11-21 18:59:43
 and a java implementation takes 70s ;)2009-11-21 18:59:51
<red> what do you think could be the practical implementation of "scan"2009-11-21 19:01:58
 i can think of lots... :)2009-11-21 19:02:34
<Shelwien> there's not much choice actually, its all limited somewhere2009-11-21 19:03:02
 1. it should work with unicode these days2009-11-21 19:03:25
 2. the scan should be "depth-first" (kinda recursive) - because directory trees are usually created that way, so its faster2009-11-21 19:04:06
 3. its best to directly use the windows API - much faster2009-11-21 19:04:25
 anyway, its not that unlikely to have to handle even millions of files/dirs these days2009-11-21 19:05:00
<red> yep2009-11-21 19:05:17
<Shelwien> and this is one of common tasks in many applications2009-11-21 19:05:26
 but i couldn't find any usable source for it2009-11-21 19:05:55
 well, 7-zip has some, but its like 100k as i said2009-11-21 19:06:09
 much too bloated2009-11-21 19:06:14
 its a common thing though2009-11-21 19:06:40
 the same applies to many common tasks2009-11-21 19:07:01
 like networking - its very easy to work with a single connection (same as files basically), but very non-trivial with many connections2009-11-21 19:08:07
<red> nice of you to share it with us2009-11-21 19:08:11
 well yes...sockets and all2009-11-21 19:08:46
<Shelwien> i tend to lose interest in things if there's no feedback ;)2009-11-21 19:08:48
 i'd say more like multithreading and all2009-11-21 19:09:02
 sockets are easy2009-11-21 19:09:08
 but posix network API is crazy2009-11-21 19:09:15
 like who would understand detecting connecting termination2009-11-21 19:09:54
 by read event with 0 bytes of actual data?2009-11-21 19:10:07
 (also, afair, "accept" was cause a dummy write event too)2009-11-21 19:10:47
 *was causing2009-11-21 19:10:56
<red> everything has its quirks.... 2009-11-21 19:11:14
<Shelwien> i guess2009-11-21 19:11:29
 just that it seems that there's still quite a lot of work for C++ programmers ;)2009-11-21 19:11:59
<red> unique is nice ..but standardization is the way to go :))2009-11-21 19:12:18
<Shelwien> its not about being unique, but about performance and features2009-11-21 19:12:57
 i don't like standardization which doesn't allow me to run a program on my hosting shell2009-11-21 19:13:27
 because that program expects to be properly installed2009-11-21 19:13:38
 and only root can do that, which i don't have2009-11-21 19:13:47
*** zapper has joined the channel2009-11-21 19:14:12
<red> that is a problem2009-11-21 19:14:21
<Shelwien> zapper: you have no patience ;) - http://ps16893.dreamhost.com/cgi-system/chantail.cgi?322009-11-21 19:14:40
<red> i've had some ..with all that win vista/win7 2009-11-21 19:14:56
<Shelwien> yeah2009-11-21 19:15:05
 unixes have even more of that ;)2009-11-21 19:15:30
<red> at least they are democratic2009-11-21 19:16:25
<Shelwien> not sure about that2009-11-21 19:16:54
 user accounts are much more limited there2009-11-21 19:17:11
 but there's still no actual security anyway2009-11-21 19:17:29
<red> you need microsofts certificates else add manifests to your program2009-11-21 19:17:37
 manifest included might cause some quirks with win xp2009-11-21 19:18:04
<Shelwien> well, you need an apple's certificate to even compile an app for iphone ;)2009-11-21 19:18:11
<red> at least they have a jailbroken thing going on2009-11-21 19:18:35
<Shelwien> sure, like i'd pay them ;)2009-11-21 19:19:01
<red> hey .. i was thinking of a gui for single file compressors2009-11-21 19:19:54
 some one in the forum was talking about them....2009-11-21 19:20:15
<Shelwien> so? there're quite a few already2009-11-21 19:20:18
 freearc is one, in fact2009-11-21 19:20:26
<red> but freearc is also a compressor2009-11-21 19:20:41
<Shelwien> not really2009-11-21 19:20:47
<red> ?2009-11-21 19:20:57
<Shelwien> it does compression, but mostly with external compressors2009-11-21 19:21:10
 well, the open source ones might be precompiled, dunno2009-11-21 19:21:28
<red> open source?2009-11-21 19:21:40
<Shelwien> but basically freearc is a haskell shell for single-file compressors2009-11-21 19:21:54
<red> bulat has builtin some of the compressors2009-11-21 19:22:01
<Shelwien> you can add any to its config2009-11-21 19:22:02
 and that use it in fa archives2009-11-21 19:22:11
<red> yes you can2009-11-21 19:22:12
 its cumbersome2009-11-21 19:22:21
 some think tailormade that only provides tar2009-11-21 19:22:51
 *thing2009-11-21 19:22:57
<Shelwien> dunno. i never tested fa actually ;)2009-11-21 19:23:32
 ...anyway, there still might be a demand for a new GUI like that2009-11-21 19:25:28
<red> well from what i gathered ..that newbie compression ethusiast did look into a lot of things2009-11-21 19:25:44
<Shelwien> but it has to be fast and light and convenient at that2009-11-21 19:26:02
 and has to support zip/7z/rar archives to boot2009-11-21 19:26:19
<red> peazip ,fa, something named coffee 2009-11-21 19:26:20
<Shelwien> yeah, though that's too dumb most of the time2009-11-21 19:26:51
 archiver is a sensitive application2009-11-21 19:27:09
<red>  i cant find that thread..in the forum now.(.will do lateron)2009-11-21 19:27:10
<Shelwien> like, you can't have it failing in some rare cases2009-11-21 19:27:39
<red> nah...i'm not thinking of a gui compressor frontend2009-11-21 19:27:49
<Shelwien> ;)2009-11-21 19:28:04
<red> something like a wrapped for commandline single file compressor2009-11-21 19:28:26
 *wrapper2009-11-21 19:28:32
<Shelwien> good too, i guess2009-11-21 19:28:48
<red> that has xml based gui2009-11-21 19:28:51
<Shelwien> but you'd still require an archive format and all2009-11-21 19:29:12
<red> so a coder and provide a gui for his commandline program2009-11-21 19:29:18
<Shelwien> and options to work with that format2009-11-21 19:29:23
 as to gui btw2009-11-21 19:29:43
 its also possible to make a web interface ;)2009-11-21 19:29:54
<red> all that gui providers is tar or gzip support2009-11-21 19:29:54
<Shelwien> like, by opening a web server at some port ;)2009-11-21 19:30:23
 well, tar is bloated, so its better to forget about it ;)2009-11-21 19:30:51
<red> yeah but internet explorer isint nice2009-11-21 19:31:05
<Shelwien> well, chrome? ;)2009-11-21 19:31:29
<red> and firefox/xulrunner is 30-40 mb2009-11-21 19:31:31
<Shelwien> yeah, but you can use any default browser anyway2009-11-21 19:31:57
<red> i could2009-11-21 19:32:12
 but what i want is a GUI that is customizable by the programmer of the compressor2009-11-21 19:32:51
 according to his needs2009-11-21 19:33:05
<Shelwien> well, the main thing imho2009-11-21 19:33:17
 would be to decide first2009-11-21 19:33:26
 whats the purpose of this whole app2009-11-21 19:33:44
<red> hmm..2009-11-21 19:33:53
 its like peazip2009-11-21 19:34:06
<Shelwien> like, is it compressing files into a single file for convenience2009-11-21 19:34:14
 or (remote) backup2009-11-21 19:34:19
<red> wait2009-11-21 19:34:30
<Shelwien> or effective transfer/storage2009-11-21 19:34:39
 it requires very different features depending on that2009-11-21 19:34:54
<red> WAIT2009-11-21 19:35:03
* Shelwien waits2009-11-21 19:35:13
 it does nothing but provides a gui some pretty buttons to click for those(users) who dont want to use commandlines2009-11-21 19:36:13
<Shelwien> i understood that, and that's why i'm talking about that ;)2009-11-21 19:36:49
 i mean, you can't just make a GUI for, say, ccm2009-11-21 19:37:06
 which would allow you to select any file and compress it2009-11-21 19:37:17
<red> the user@ encode forum wanted something so he could pipe the2009-11-21 19:37:34
<Shelwien> i mean, how that can be used?2009-11-21 19:37:35
<red> progress on screen and provide file browser2009-11-21 19:37:51
<Shelwien> sure, but i'm saying that its troublesome2009-11-21 19:37:57
 also you just reminded me about BWT codecs2009-11-21 19:38:07
 and BWT codecs are special2009-11-21 19:38:30
<red> hmmm...2009-11-21 19:38:32
<Shelwien> as they're very blockwise2009-11-21 19:38:40
 and usually, while it processes a block (commonly quite large)2009-11-21 19:39:08
 you don't have any progress2009-11-21 19:39:13
<red> this is difficult 2009-11-21 19:39:35
<Shelwien> yeah, i can imagine some alternative approaches though2009-11-21 19:40:08
<red> ok then...we'll talk later when i have some code to show for it2009-11-21 19:40:10
<Shelwien> ...2009-11-21 19:40:20
 i'm saying that its not about coding really ;)2009-11-21 19:40:31
<red> its not2009-11-21 19:40:40
<Shelwien> you just have to know precisely what're you trying to do ;)2009-11-21 19:40:45
<red> of course2009-11-21 19:40:54
<Shelwien> at least, common use cases2009-11-21 19:40:57
<red> indeed i have given some thoughtr2009-11-21 19:41:17
 shelwin makes a file compression program2009-11-21 19:42:12
 he can only compress a single file with it2009-11-21 19:42:39
 his program is commandline(of course)2009-11-21 19:43:12
 me : i have basic gui of common function a file archiver does2009-11-21 19:44:12
 i provide an editor(program) for the gui and a way to customize the attibutes2009-11-21 19:45:02
 now shelwin can distribute his program (compressor) and use my editor to make a GUI for it2009-11-21 19:45:56
 he has a file archiver2009-11-21 19:46:24
<Shelwien> yeah, i got that, but there're still too many questions/options2009-11-21 19:46:56
<red> what do you think?2009-11-21 19:46:56
  hit me...2009-11-21 19:47:12
<Shelwien> 1. is GUI supposed to be configured for a single compressor or would be able to support multiple at once?2009-11-21 19:47:41
<red> totally up to you2009-11-21 19:48:07
 at once?2009-11-21 19:48:38
<Shelwien> 2. would the compressor interface use pipes, or files; would it process the compressor console messages? or what about an option of hooking the system APIs and/or loading the compressor by yourself?2009-11-21 19:49:57
 3. would it only provide options to create and archive and extract it, or something else maybe? (update)2009-11-21 19:50:43
 4. filters (precomp?) and combining multiple algorithms (like LZ+ppmd in rar)?2009-11-21 19:51:23
 5. SFX?2009-11-21 19:51:50
<red> 2. it could have a separate thing(methods) for dealing with messeages 2009-11-21 19:51:57
 no hooking sys api...2009-11-21 19:52:13
<Shelwien> why? ;) its convenient2009-11-21 19:52:24
<red> 3. only the bare minimum2009-11-21 19:52:26
 4 why not2009-11-21 19:52:35
 5 could be2009-11-21 19:52:44
 i'm trying to make this thing to help ....not make a new compressor2009-11-21 19:53:30
<Shelwien> yeah, but there's still a question of use cases then2009-11-21 19:54:11
 what i can imagine could be useful2009-11-21 19:54:30
 is an archiver GUI for general use2009-11-21 19:54:43
<red> least common amongst them2009-11-21 19:54:43
<Shelwien> with zip/rar/7z support as i said2009-11-21 19:54:53
 where advanced users would be able to add existing compressors2009-11-21 19:55:30
<red> there will be no support for any compressor2009-11-21 19:55:54
 from my side2009-11-21 19:56:10
 anyone can add in whatever they want2009-11-21 19:56:24
<Shelwien> well, note that compressor developers won't do that, and won't try to make it compatible (by providing support for using pipes or whatever)2009-11-21 19:57:31
<red> addins will be through and xml file or some other easily configurable method2009-11-21 19:57:37
<Shelwien> don't you think that compressor developer can just use any IDE to do that instead?2009-11-21 19:58:45
<red> well i thought most has trouble with gui and stuff and reinventing the wheel everytime2009-11-21 19:59:09
<Shelwien> we do have troubles with gui, yeah, but we don't really reinvent anything2009-11-21 19:59:33
 nobody needs GUI really2009-11-21 19:59:42
 (i mean developers)2009-11-21 19:59:51
<red> well those with multiple compressors to their name dont2009-11-21 19:59:55
<Shelwien> no, i meant something different2009-11-21 20:00:17
<red> reinvent....they reuse cod2009-11-21 20:00:19
 e2009-11-21 20:00:22
 i mean something diffrent too2009-11-21 20:00:36
 *meant2009-11-21 20:00:42
 gui thing by me will mostly be for endusers2009-11-21 20:01:30
 convenience 2009-11-21 20:01:44
<Shelwien> yeah, that's my point too2009-11-21 20:01:51
<red> but ....2009-11-21 20:02:08
<Shelwien> which means that you have to provide ways to handle compressors which weren't specially built for your shell2009-11-21 20:02:27
<red> lets see.. i need some incentive for devlopers....hmmm2009-11-21 20:03:02
<Shelwien> and i'd repeat the question again: wtf users are supposed to do with that GUI?2009-11-21 20:03:21
<red> use it for compression and decompression2009-11-21 20:04:12
 files/ folders2009-11-21 20:04:24
<Shelwien> that's too unspecific2009-11-21 20:04:44
<red> well for eg2009-11-21 20:04:56
 if you add 7zip support2009-11-21 20:05:12
<Shelwien> you see, compressing texts, and images, and videos, and dvdisos are all different tasks in fact ;)2009-11-21 20:05:26
<red> *that are handled by compressors....2009-11-21 20:06:00
<Shelwien> not quite2009-11-21 20:06:16
<red> ? how so2009-11-21 20:06:22
<Shelwien> you see, you can't compress dvdisos with paq8 ;)2009-11-21 20:06:25
 even if you embed it into your gui ;)2009-11-21 20:06:37
 and most of the usual stuff is precompressed already2009-11-21 20:07:16
 like music/video/images/program installs2009-11-21 20:07:27
 so your GUI either has to contain something like precomp in it2009-11-21 20:07:44
 or whether won't be any differences from zip/rar for practical use2009-11-21 20:08:07
 *or there2009-11-21 20:08:13
 ...do you know precomp at all btw? ;)2009-11-21 20:08:39
<red> yep2009-11-21 20:08:47
 nifty little thing2009-11-21 20:09:15
<Shelwien> not really, but there're no alternatives atm2009-11-21 20:09:32
 anyway, my point is that there're some specific use cases for archiver GUIs2009-11-21 20:10:25
<red> yes2009-11-21 20:10:26
<Shelwien> which might require some work2009-11-21 20:10:32
<red> indeed some heavy work2009-11-21 20:10:47
<Shelwien> like remote backup (which i'm working on atm btw)2009-11-21 20:10:47
<red> multiplexing2009-11-21 20:11:16
<Shelwien> or compressed storage with (relatively) random access2009-11-21 20:11:34
 like resources in games etc2009-11-21 20:11:42
<red> i love "pc rips"2009-11-21 20:12:00
 :)2009-11-21 20:12:06
<Shelwien> sure, but these days2009-11-21 20:12:18
 unfortunately, it can't be done just by using a single compressor2009-11-21 20:12:35
<red> yep those bat files2009-11-21 20:12:50
<Shelwien> also, don't forget that there're specialized codecs2009-11-21 20:13:02
 like mp3zip/soundslimmer or optimfrog, or bmf2009-11-21 20:13:25
<red> yes2009-11-21 20:14:00
 ogg is the fav...2009-11-21 20:14:14
 lossy compression in rips there days2009-11-21 20:14:44
 *these2009-11-21 20:14:49
<Shelwien> whatever, there's no sense to lossily convert all the mp3s to ogg ;)2009-11-21 20:14:52
 but there're lossless compressors for mp3s ;)2009-11-21 20:15:06
<red> i"ll talk to you later2009-11-21 20:16:44
 need sleep2009-11-21 20:17:15
<Shelwien> ;)2009-11-21 20:17:37
<red> i"m at encode i'm the one with the turtle ....2009-11-21 20:17:59
 lets see if you know who i am2009-11-21 20:18:12
 ..bye2009-11-21 20:18:18
*** red has left the channel2009-11-21 20:18:27
<zapper> i wonder who it could be?2009-11-21 20:33:42
<Shelwien> i don't know, but not that i care ;)2009-11-21 20:34:07
<zapper> me neither ;) LoL2009-11-21 20:34:38
<Shelwien> yeah, i don't know who you are either ;)2009-11-21 20:35:14
 !grep zapper2009-11-21 20:36:15
 but you appearing and quitting after a minute was fun anyway ;)2009-11-21 20:37:04
 you see, its now how irc works ;)2009-11-21 20:37:10
 when you join, you don't normally see anything that was said before ;)2009-11-21 20:37:43
<zapper> as i see it anyone can join this chat as say 'encode' and you never know if it really is encode2009-11-21 20:39:36
<Shelwien> in a way2009-11-21 20:39:53
<zapper> how?2009-11-21 20:40:00
<Shelwien> but its not popular enough for real fakes to appear2009-11-21 20:40:09
 and then, its possible to register a nick on irc2009-11-21 20:40:17
 then you would be able to use it only if you know a password2009-11-21 20:40:45
 ...i don't think its compatible with that webchat though2009-11-21 20:41:22
<zapper> i really dont like the way people can use any nick they like her. 2009-11-21 20:41:45
 e2009-11-21 20:41:48
 can anything be done about this?2009-11-21 20:42:17
<Shelwien> you mean, with other people?2009-11-21 20:42:36
 but on the forum their nicks are not any more comprehensible actually2009-11-21 20:43:10
 usually you only know who it is from programs they post etc2009-11-21 20:43:30
<zapper> but at least they need to sign in at the forum2009-11-21 20:44:01
 btw i enjoyed your chat with big red2009-11-21 20:44:21
<Shelwien> do you know the log url btw?2009-11-21 20:45:14
 i mean this: http://ps16893.dreamhost.com/log/2009-11-21 20:45:46
 you missed some more people lately ;)2009-11-21 20:46:11
<zapper> just looking at the log files 2009-11-21 20:48:01
 LoL2009-11-21 20:48:04
 i dont really like to join in these live chats myself but usually enjoy it when there's some interesting chat in progress2009-11-21 20:50:00
<Shelwien> well, its more like "talk to shelwien" than compression channel actually ;)2009-11-21 20:51:20
<zapper> do you have to treat these chats as 'topic'/'off topic' like at the forum 2009-11-21 20:51:29
 are the rules strict here?2009-11-21 20:51:38
<Shelwien> not really2009-11-21 20:51:49
 there's some bodybuilding chat here2009-11-21 20:52:06
 toffer and lasse went about it for hours ;)2009-11-21 20:52:16
<zapper> ah toffer's here. this should be an interesting chat2009-11-21 20:52:51
<Shelwien> also sami appears occasionally2009-11-21 20:53:11
<zapper> that's the way i see this chat2009-11-21 20:53:18
<Shelwien> well, it was almost like i'd want to see it when ccm decompiling project was active2009-11-21 20:53:56
<zapper> i wish someone else would join the chat so i can sit back and enjoy again :)2009-11-21 20:55:31
 im not much of a chat person myself2009-11-21 20:55:45
 as you probably guessed LoL2009-11-21 20:55:58
<Shelwien> asmodean and pinc here are like that too2009-11-21 20:56:46
 but at least they lurk in the channel ;)2009-11-21 20:56:55
 adding numbers to the crowd ;)2009-11-21 20:57:15
<zapper> why do you think the forum has been so quiet lately?2009-11-21 20:57:35
<Shelwien> some seasonal thing maybe2009-11-21 20:57:57
 toffer has his thesis etc2009-11-21 20:58:06
 many are busy with work too2009-11-21 20:58:20
<zapper> maybe? but i don't like it2009-11-21 20:58:21
<Shelwien> sure, but i can't really fix it by myself ;)2009-11-21 20:58:41
<zapper> nor i2009-11-21 20:58:54
 :)2009-11-21 20:58:59
<Shelwien> you can in fact ;)2009-11-21 20:59:06
<zapper> i wish i could :)2009-11-21 20:59:18
<Shelwien> i mean, i have 600 posts in that forum as it is2009-11-21 20:59:51
 so not that i can post much more2009-11-21 21:00:27
<zapper> same here2009-11-21 21:00:50
 the forum seems to get reaaly interesting and busy then it just seems to come to a stop2009-11-21 21:01:48
 dead stop that is :)2009-11-21 21:02:00
<Shelwien> well, the program maintenance threads save it in a way2009-11-21 21:02:25
<zapper> no one writing a super powerful compressor lately either LoL2009-11-21 21:02:46
<Shelwien> well, there're a few paq8 branches ;)2009-11-21 21:03:14
<zapper> yea but...2009-11-21 21:03:55
<Shelwien> well, compression became kinda specialized2009-11-21 21:04:24
 as i tried to explain to "red" here2009-11-21 21:04:46
 nobody really needs a rar equivalent with different codecs2009-11-21 21:05:08
<zapper> i think many members are waiting for the next magic compressor like CCM was at the time2009-11-21 21:05:18
<Shelwien> well, i still don't understand what's so special about ccm even now2009-11-21 21:05:51
 !grep results.txt2009-11-21 21:05:59
<zapper> did you say sami sometimes joins this chat?2009-11-21 21:06:14
<Shelwien> yeah2009-11-21 21:06:19
 but i meant this:2009-11-21 21:06:58
 32.235s 32.359s ccm 1.30c 52009-11-21 21:06:59
 28.187s 29.515s 22004883 ccm_sh1d9e2009-11-21 21:07:08
<zapper> ccm is considered special because it gives paq like compression but is quite fast. thats the way i understand it.2009-11-21 21:07:26
<Shelwien> its not even nearly paq-like2009-11-21 21:07:41
 well, its only paq-line in the sense that its CM too2009-11-21 21:07:54
<zapper> i understand that but its the way its perceived to be paq-like2009-11-21 21:08:08
<Shelwien> well, its some weird fashion thing imho2009-11-21 21:08:27
<zapper> it basically the way members see it that make it special2009-11-21 21:08:41
 if people think is junk, then it will be. even if its good.2009-11-21 21:09:08
 thats the way i undestand it anyway2009-11-21 21:09:32
<Shelwien> yeah, kinda, but its more about whether they see it at all2009-11-21 21:09:34
 like i have a ccm source here and nobody cares ;)2009-11-21 21:09:57
 and that version is 10% faster even ;)2009-11-21 21:10:09
<zapper> i care but people dont want to tread on toffers toes with this2009-11-21 21:10:29
<Shelwien> dunno really2009-11-21 21:11:04
<zapper> he's a well respected member of the forum2009-11-21 21:11:13
<Shelwien> there're quite a lot of different things to do2009-11-21 21:11:15
<zapper> is all about the people2009-11-21 21:11:49
<Shelwien> well, toffer is basically the only person who discusses compression theory with me ;)2009-11-21 21:11:54
 but that doesn't mean that you can't make a compressor which would work better than his2009-11-21 21:12:34
 as he only concentrates on specific aspects of the model2009-11-21 21:12:54
 ...same with me, i guess2009-11-21 21:13:08
<zapper> btw i meant chris not toffer sorry. toffer is not the author of ccm. sorry2009-11-21 21:13:11
<Shelwien> toffer's cmm is not that bad either ;)2009-11-21 21:13:33
<zapper> true2009-11-21 21:15:11
 will come back here later. my wife has just asked me to do a little job here :)2009-11-21 21:15:56
<Shelwien> ;)2009-11-21 21:16:03
 !next2009-11-21 21:16:05