*** Shelwien has left the channel | 2009-11-19 02:11:31 |
*** Shelwien has joined the channel | 2009-11-19 02:11:35 |
*** pinc has joined the channel | 2009-11-19 07:53:13 |
*** pinc has left the channel | 2009-11-19 08:57:07 |
*** pinc has joined the channel | 2009-11-19 09:26:30 |
*** toffer has joined the channel | 2009-11-19 10:26:19 |
*** pinc|mirror has joined the channel | 2009-11-19 12:41:41 |
*** pinc has left the channel | 2009-11-19 12:41:41 |
*** pinc|mirror has left the channel | 2009-11-19 12:46:09 |
*** pinc has joined the channel | 2009-11-19 12:48:37 |
*** pinc|mirror has joined the channel | 2009-11-19 12:57:11 |
*** pinc has left the channel | 2009-11-19 12:57:11 |
*** moisesmcardona has joined the channel | 2009-11-19 13:11:37 |
*** moisesmcardona has left the channel | 2009-11-19 13:13:43 |
*** pinc|mirror has left the channel | 2009-11-19 17:04:24 |
*** Shelwien has left the channel | 2009-11-19 17:51:02 |
*** pinc has joined the channel | 2009-11-19 17:59:52 |
*** toffer has left the channel | 2009-11-19 19:55:40 |
*** pinc has left the channel | 2009-11-19 20:47:35 |
*** Shelwien has joined the channel | 2009-11-19 23:47:24 |
*** Shelwien has left the channel | 2009-11-20 06:30:20 |
*** Shelwien has joined the channel | 2009-11-20 06:33:33 |
*** Shelwien has left the channel | 2009-11-20 06:42:50 |
*** Shelwien has joined the channel | 2009-11-20 07:34:38 |
*** chornobyl has joined the channel | 2009-11-20 10:07:47 |
<Shelwien> | 2009-11-20 10:26:16 weintainypeIo...> ...please where can I buy a unicorn? | 2009-11-20 10:34:03 |
| this spam is funny | 2009-11-20 10:34:14 |
| http://ctxmodel.net/rem.pl?8 | 2009-11-20 10:34:23 |
<chornobyl> | lot of it | 2009-11-20 10:34:43 |
* Shelwien is installing MS SQL Server 2008. To investigate its backup compression | 2009-11-20 10:35:45 |
*** toffer has joined the channel | 2009-11-20 10:54:10 |
<toffer> | hi | 2009-11-20 10:57:34 |
<Shelwien> | hi | 2009-11-20 10:57:42 |
<chornobyl> | hello | 2009-11-20 10:57:47 |
<toffer> | i finished the slides for my presentation this morning at 4 - feeling a bit tired now | 2009-11-20 10:59:37 |
| any news, meanwhile? | 2009-11-20 10:59:52 |
<Shelwien> | not much. i'm writing a new filesystem scanner | 2009-11-20 11:00:43 |
| and all this stuff with unicode, codepages, and windows API | 2009-11-20 11:01:14 |
<toffer> | you've already written one, don't you?! | 2009-11-20 11:01:15 |
<Shelwien> | goes quite slow | 2009-11-20 11:01:20 |
| well, i'm writing a new one now ;) | 2009-11-20 11:01:34 |
| in hope that it would be even better ;) | 2009-11-20 11:01:48 |
| also this one is for my backup engine | 2009-11-20 11:02:21 |
<toffer> | good luck than | 2009-11-20 11:02:41 |
| gonna have some breakfast now | 2009-11-20 11:02:46 |
| btw did i already mention that my time series predictor outpreformed the fraunhofer one? | 2009-11-20 11:03:03 |
<Shelwien> | yeah | 2009-11-20 11:03:14 |
| that's kinda expected though ;) | 2009-11-20 11:03:23 |
<toffer> | self-praise sucks :D | 2009-11-20 11:03:47 |
| and i think when i've got my spare time back again i'll modify m1 to get into the ltcb top ten :D | 2009-11-20 11:04:59 |
| somehow lpaq shows better performance when dropping models (!) | 2009-11-20 11:05:21 |
<Shelwien> | if mean if you remove models from lpaq? | 2009-11-20 11:05:55 |
<toffer> | yes | 2009-11-20 11:06:00 |
| order 3 is useless for example | 2009-11-20 11:06:05 |
| almost no change | 2009-11-20 11:06:07 |
| to test some of my ideas | 2009-11-20 11:06:26 |
| i modified it to use orders 248, match and word | 2009-11-20 11:06:58 |
| it produces nearly the same compression | 2009-11-20 11:07:08 |
| on the ltcb | 2009-11-20 11:07:11 |
<Shelwien> | well, its special | 2009-11-20 11:07:27 |
<toffer> | thus if you'd add a dictionary to that | 2009-11-20 11:07:27 |
| you'd get a top 10 entry | 2009-11-20 11:07:36 |
| i know | 2009-11-20 11:07:42 |
| it's not universal | 2009-11-20 11:07:48 |
| still these chained sses are not useful - they mostly eat up cpu time | 2009-11-20 11:08:04 |
<Shelwien> | yeah, Matt seems to still have troubles with SSE handling | 2009-11-20 11:08:59 |
<toffer> | i mean just 4-5 specialized models are enough to produce very good compression | 2009-11-20 11:09:15 |
| and one or two of these can be implemenetd via direct lookup | 2009-11-20 11:10:56 |
| guess i'll add a modification to have one or two "low order models" for speed | 2009-11-20 11:11:58 |
*** Shelwien has left the channel | 2009-11-20 11:13:49 |
*** Shelwien has joined the channel | 2009-11-20 11:14:02 |
| and i found that sse for mixing is faster than logistic mixing - well if it isn't vectorized | 2009-11-20 11:14:44 |
<Shelwien> | yeah | 2009-11-20 11:15:18 |
<toffer> | since prediction is just a lookup | 2009-11-20 11:15:28 |
<Shelwien> | i think Matt didn't ever use SSE2+ too | 2009-11-20 11:15:31 |
| though i guess it should be possible to add some logistic extrapolation to SSE | 2009-11-20 11:16:02 |
<toffer> | do you mean to a sse counter or via using input probs. explicitly_ | 2009-11-20 11:17:03 |
| and something else - what kind of probability estimation is shkarin use? i remember some weird stuff "substract mean" or something like that in a comment? | 2009-11-20 11:19:33 |
<Shelwien> | well, there's sse probability estimation | 2009-11-20 11:19:42 |
| and even if you apply SSE in logistic space | 2009-11-20 11:19:59 |
| its still possible to add some multiplier after SSE | 2009-11-20 11:20:17 |
<toffer> | multiplier in the sense of rescaling? | 2009-11-20 11:20:56 |
<Shelwien> | and Shkarin mostly uses integer frequencies everywhere | 2009-11-20 11:21:04 |
| but as to means etc - why not | 2009-11-20 11:21:20 |
| i mean, its a kind of linear extrapolation etc | 2009-11-20 11:22:17 |
| ...and multiplier like in sq(C*st(p)) | 2009-11-20 11:22:49 |
<toffer> | not that important. it just came to my mind these days and i didn't know what that would be intended for | 2009-11-20 11:22:51 |
| yeah so rescaling | 2009-11-20 11:22:58 |
<Shelwien> | i'd say "rescaling" is something different ;) | 2009-11-20 11:23:20 |
<toffer> | C>1 ? | 2009-11-20 11:23:22 |
| well it is the same | 2009-11-20 11:23:29 |
<Shelwien> | probably, but not necessarily | 2009-11-20 11:23:40 |
| i think i had it tuned to <1 somewhere too | 2009-11-20 11:23:55 |
<toffer> | but the steps will be larger near 0.5 and smaller near 0/1 | 2009-11-20 11:23:58 |
| step = delta sq(..) vs. delta C | 2009-11-20 11:24:24 |
<Shelwien> | dunno, i have 15 bit precision in logistic domain too | 2009-11-20 11:24:31 |
| and C* is done in fixed point obviously | 2009-11-20 11:24:55 |
| C*s>>15 or something | 2009-11-20 11:25:10 |
<toffer> | well that's clear | 2009-11-20 11:25:24 |
| i meant comparing the effect in probability domain | 2009-11-20 11:25:35 |
<Shelwien> | should be smooth enough, dunno | 2009-11-20 11:26:12 |
| and anyway, its probably possible to invent a better extrapolation for SSE | 2009-11-20 11:28:43 |
| with likelihoods etc | 2009-11-20 11:28:52 |
| there're two mixed points, but the logic is somewhat different from mixer | 2009-11-20 11:30:07 |
<toffer> | two mixed points? | 2009-11-20 11:31:57 |
<Shelwien> | ah, i forgot | 2009-11-20 11:32:07 |
| i'm talking about interpolated 1D SSE obviously | 2009-11-20 11:32:23 |
<toffer> | sq(C*st(p)) seems to be 1 dimensional :) | 2009-11-20 11:34:13 |
| still i don't know what "two mixed points" should be | 2009-11-20 11:34:33 |
<Shelwien> | well, SSE can be interpolated | 2009-11-20 11:35:28 |
<toffer> | yes | 2009-11-20 11:35:53 |
| but why'd you want to interpolate "sq(C*st(p))" | 2009-11-20 11:36:13 |
<Shelwien> | so there'd be some sse[q]*r+sse[q+1]*(1-r) | 2009-11-20 11:36:31 |
<toffer> | ok | 2009-11-20 11:36:36 |
| ah | 2009-11-20 11:36:42 |
| now i know what you actually mean | 2009-11-20 11:36:48 |
<Shelwien> | i meant two unrelated things actually | 2009-11-20 11:37:01 |
<toffer> | erm? | 2009-11-20 11:37:23 |
<Shelwien> | 1. that sq(C*SSE(p)) can be a used after SSE as a posterior extrapolation | 2009-11-20 11:38:04 |
<toffer> | afaiu you want to replace the predictions in a "bin"-based sse (prob -> bin, weight) with refined predictions "sse[q]*r+sse[q+1]*(1-r)" | 2009-11-20 11:38:35 |
<Shelwien> | 2. that interpolated SSE is kinda like mixing, but its logic is completely different, so if we'd think about it, the underlying model etc might be different too | 2009-11-20 11:38:46 |
| so different likelihoods etc | 2009-11-20 11:39:01 |
| and thus different mixing in likelihood domain | 2009-11-20 11:39:19 |
<toffer> | wouldn't it be best to use any of these mappings after sse | 2009-11-20 11:40:22 |
| mh | 2009-11-20 11:40:43 |
| well these mappings are satic? | 2009-11-20 11:40:48 |
| sq(C*SSE(p)), C is constant during processing | 2009-11-20 11:41:04 |
<Shelwien> | yeah | 2009-11-20 11:41:45 |
<toffer> | at least i now know what you're meaning with sse and mixing have in common | 2009-11-20 11:41:56 |
<Shelwien> | well, i'm actually trying to say | 2009-11-20 11:42:31 |
| that interpolated SSE in logistic domain | 2009-11-20 11:42:42 |
| might have to be completely different from paq mixer | 2009-11-20 11:42:51 |
| as it can't be nicely approximated by a switching model like mixer | 2009-11-20 11:43:34 |
<toffer> | sure? selecting a single bin works for me | 2009-11-20 11:44:04 |
| i mean the nearest | 2009-11-20 11:44:08 |
<Shelwien> | yeah, whatever | 2009-11-20 11:44:16 |
<toffer> | r<0.5 ? 0 : 1 | 2009-11-20 11:44:19 |
| r<0.5 ? left : right | 2009-11-20 11:44:29 |
| or something like that | 2009-11-20 11:44:36 |
<Shelwien> | i know, been there | 2009-11-20 11:44:40 |
<toffer> | but i think i understood what you said | 2009-11-20 11:44:49 |
<Shelwien> | i've only used linear interpolated SSE (and SSE2) until now | 2009-11-20 11:45:28 |
<toffer> | it works even quite well for 2d sse | 2009-11-20 11:45:55 |
<Shelwien> | but i also did some experiments with nonlinear mapping for the probability | 2009-11-20 11:45:58 |
<toffer> | i'm using it all the time | 2009-11-20 11:46:00 |
<Shelwien> | and it helped a little, even though that was some ad hoc nonlinearity, not anything like stretch | 2009-11-20 11:46:55 |
| although, in fact | 2009-11-20 11:48:07 |
| my interpolated SSE works good enough as is too, somehow | 2009-11-20 11:48:22 |
| ash shows like 202k on book1 | 2009-11-20 11:48:46 |
| and there're no word models or anything like that | 2009-11-20 11:49:00 |
| ... | 2009-11-20 11:50:50 |
| and again, what i mean is | 2009-11-20 11:50:59 |
| that my explanation for paq mixer | 2009-11-20 11:51:07 |
| was based on likelihoods for an approximate model | 2009-11-20 11:51:36 |
| looking like two static probability estimation, switched with a static probability | 2009-11-20 11:52:02 |
| *estimations | 2009-11-20 11:52:15 |
| and that obviously doesn't apply to SSE | 2009-11-20 11:53:04 |
<toffer> | i think interpreting sse like that might not be right. since the "mixing" is just linear interpolation. there might be other solutions which work better | 2009-11-20 11:54:16 |
<Shelwien> | like what? | 2009-11-20 11:54:41 |
<toffer> | sse[q]*r+sse[q+1]*(1-r) | 2009-11-20 11:55:00 |
<Shelwien> | you quantize the input probability | 2009-11-20 11:55:04 |
| and that leaves a remainder | 2009-11-20 11:55:21 |
<toffer> | yeah | 2009-11-20 11:55:25 |
| i know | 2009-11-20 11:55:26 |
<Shelwien> | but, the same likelihood approach apply here too | 2009-11-20 11:55:41 |
| estimating likelihoods for n0+1 case and for n1+1 case | 2009-11-20 11:55:58 |
| and afaik | 2009-11-20 11:57:00 |
| that would give a significantly different function from logistic mixing | 2009-11-20 11:57:15 |
| though obviously with log/exp too | 2009-11-20 11:57:30 |
<toffer> | sorry have to leave now for gym & travelling home | 2009-11-20 12:04:25 |
<Shelwien> | ok, i'd sleep anyway | 2009-11-20 12:04:44 |
<toffer> | so "good night" - or "good day" :D | 2009-11-20 12:05:58 |
<Shelwien> | ;) | 2009-11-20 12:06:10 |
*** toffer has left the channel | 2009-11-20 12:07:40 |
*** chornobyl has left the channel | 2009-11-20 12:57:13 |
*** zapper has joined the channel | 2009-11-20 16:36:40 |
* zapper slaps complogger around a bit with a large fishbot | 2009-11-20 16:37:17 |
* zapper slaps Shelwien around a bit with a large fishbot | 2009-11-20 16:38:05 |
* zapper slaps asmodean around a bit with a large fishbot | 2009-11-20 16:38:20 |
*** zapper has left the channel | 2009-11-20 16:38:30 |
*** Shelwien has left the channel | 2009-11-21 11:13:09 |
*** Guest9968193 has joined the channel | 2009-11-21 11:13:13 |
*** pinc has joined the channel | 2009-11-21 17:28:27 |
*** red has joined the channel | 2009-11-21 18:20:20 |
<red> | hey all | 2009-11-21 18:20:29 |
<Shelwien> | hi | 2009-11-21 18:20:57 |
<red> | hi | 2009-11-21 18:21:43 |
| just read your "Directory scanning in windows" post ..nice | 2009-11-21 18:22:37 |
<Shelwien> | well, i'd say its quite troublesome | 2009-11-21 18:23:34 |
<red> | informative..nonetheless | 2009-11-21 18:24:12 |
<Shelwien> | did you see the benchmarks? ;) | 2009-11-21 18:24:26 |
*** kj has joined the channel | 2009-11-21 18:26:16 |
*** kj has left the channel | 2009-11-21 18:26:21 |
<red> | what benchmarks ?? | 2009-11-21 18:27:17 |
*** pinc has left the channel | 2009-11-21 18:27:18 |
<Shelwien> | there's some in http://shelwien.googlepages.com/sami_template_v2.rar | 2009-11-21 18:27:56 |
| ah, also this: http://compressionratings.com/s_scan.html | 2009-11-21 18:29:57 |
<red> | this is good | 2009-11-21 18:31:06 |
<Shelwien> | hope so | 2009-11-21 18:31:39 |
| unfortunately only my implementation supports unicode | 2009-11-21 18:31:58 |
| ...also, there was a java version too | 2009-11-21 18:32:52 |
| only twice slower than Sami's STL ;) | 2009-11-21 18:33:02 |
<red> | i've always seen posts by you that have a code snippet or benchmarks ...that really helps thank you | 2009-11-21 18:33:34 |
<Shelwien> | well, i'm kinda trying to start some discussion with these | 2009-11-21 18:35:11 |
| no luck most of the time, though ;) | 2009-11-21 18:35:18 |
| for example, there's also a quirk like this (in directory scanning): | 2009-11-21 18:37:17 |
| if we use "." or ".." as a base path | 2009-11-21 18:37:33 |
*** red_ has joined the channel | 2009-11-21 18:37:44 |
<red_> | back | 2009-11-21 18:38:13 |
<Shelwien> | the actual directory name would be substituted instead of "." part | 2009-11-21 18:38:34 |
| check http://ps16893.dreamhost.com/cgi-system/chantail.cgi?32 if you missed anything | 2009-11-21 18:38:47 |
| so, if we make an archiver using my implementation | 2009-11-21 18:39:24 |
*** red has left the channel | 2009-11-21 18:39:41 |
| it would probably add the current directory with a name into archive, if we'd start it using "." | 2009-11-21 18:40:02 |
| while eg. rar doesn't do that - there it works the same as .\* | 2009-11-21 18:40:54 |
| so which is better? ;) | 2009-11-21 18:41:19 |
<red> | hmmm.... | 2009-11-21 18:41:45 |
| i really dont know? | 2009-11-21 18:44:23 |
<Shelwien> | that's the problem ;) | 2009-11-21 18:44:43 |
| because rar's behavior is much harder to implement actually | 2009-11-21 18:45:06 |
| because both FindFirst and GetFullPathName would convert that to a real directory name | 2009-11-21 18:45:53 |
| so we have to modify the base path from "." to ".\*" before converting it into canonical form | 2009-11-21 18:46:34 |
| but then, there can be stuff like "C:." | 2009-11-21 18:47:06 |
| or something like "\\?\C:\1\." even | 2009-11-21 18:47:21 |
| basically, that's like rewriting that GetFullPathName | 2009-11-21 18:47:53 |
| ...i really wonder why system APIs always have to be crazy... | 2009-11-21 18:48:26 |
<red> | wouldnt something like regular expression help with this sort of this | 2009-11-21 18:49:02 |
<Shelwien> | don't forget about unicode ;) | 2009-11-21 18:49:19 |
| well, i guess it would, after conversion to utf8 or something | 2009-11-21 18:49:58 |
| and kinda that's why people don't write anything in plain C/C++ anymore | 2009-11-21 18:50:42 |
| but then we get bloated programs | 2009-11-21 18:51:05 |
<red> | a temporary thing /a method that invokes the "regular ex" when some troublesome "pathname" comes up | 2009-11-21 18:52:18 |
| else it goes on to process like it does currently | 2009-11-21 18:53:07 |
<Shelwien> | yeah, but as i said, there're no regexps in plain C++ - it comes as a separate library | 2009-11-21 18:54:03 |
| and there's no build-in string type actually | 2009-11-21 18:54:16 |
| *built-in | 2009-11-21 18:54:25 |
<red> | i understand | 2009-11-21 18:55:08 |
<Shelwien> | but my scanner in C++ can be like 4k executable (depending on compiler options) | 2009-11-21 18:55:09 |
| and equivalent implementation in perl, for example | 2009-11-21 18:55:42 |
| would require like 3 dlls to distribute with it | 2009-11-21 18:55:55 |
<red> | but what good a 4k scanner it it cant work ? | 2009-11-21 18:56:11 |
| *if | 2009-11-21 18:56:18 |
<Shelwien> | err, but it can | 2009-11-21 18:56:27 |
| even though it might behave a little different from what other programs do | 2009-11-21 18:56:57 |
| but on other hand | 2009-11-21 18:57:06 |
| most of these "other programs" might have other - and more troublesome - problems | 2009-11-21 18:57:25 |
| like not understanding national filenames etc | 2009-11-21 18:58:17 |
| and there's a question of time/speed too | 2009-11-21 18:58:59 |
<red> | yes | 2009-11-21 18:59:22 |
<Shelwien> | like, my scan5 takes 4s to scan some directory | 2009-11-21 18:59:43 |
| and a java implementation takes 70s ;) | 2009-11-21 18:59:51 |
<red> | what do you think could be the practical implementation of "scan" | 2009-11-21 19:01:58 |
| i can think of lots... :) | 2009-11-21 19:02:34 |
<Shelwien> | there's not much choice actually, its all limited somewhere | 2009-11-21 19:03:02 |
| 1. it should work with unicode these days | 2009-11-21 19:03:25 |
| 2. the scan should be "depth-first" (kinda recursive) - because directory trees are usually created that way, so its faster | 2009-11-21 19:04:06 |
| 3. its best to directly use the windows API - much faster | 2009-11-21 19:04:25 |
| anyway, its not that unlikely to have to handle even millions of files/dirs these days | 2009-11-21 19:05:00 |
<red> | yep | 2009-11-21 19:05:17 |
<Shelwien> | and this is one of common tasks in many applications | 2009-11-21 19:05:26 |
| but i couldn't find any usable source for it | 2009-11-21 19:05:55 |
| well, 7-zip has some, but its like 100k as i said | 2009-11-21 19:06:09 |
| much too bloated | 2009-11-21 19:06:14 |
| its a common thing though | 2009-11-21 19:06:40 |
| the same applies to many common tasks | 2009-11-21 19:07:01 |
| like networking - its very easy to work with a single connection (same as files basically), but very non-trivial with many connections | 2009-11-21 19:08:07 |
<red> | nice of you to share it with us | 2009-11-21 19:08:11 |
| well yes...sockets and all | 2009-11-21 19:08:46 |
<Shelwien> | i tend to lose interest in things if there's no feedback ;) | 2009-11-21 19:08:48 |
| i'd say more like multithreading and all | 2009-11-21 19:09:02 |
| sockets are easy | 2009-11-21 19:09:08 |
| but posix network API is crazy | 2009-11-21 19:09:15 |
| like who would understand detecting connecting termination | 2009-11-21 19:09:54 |
| by read event with 0 bytes of actual data? | 2009-11-21 19:10:07 |
| (also, afair, "accept" was cause a dummy write event too) | 2009-11-21 19:10:47 |
| *was causing | 2009-11-21 19:10:56 |
<red> | everything has its quirks.... | 2009-11-21 19:11:14 |
<Shelwien> | i guess | 2009-11-21 19:11:29 |
| just that it seems that there's still quite a lot of work for C++ programmers ;) | 2009-11-21 19:11:59 |
<red> | unique is nice ..but standardization is the way to go :)) | 2009-11-21 19:12:18 |
<Shelwien> | its not about being unique, but about performance and features | 2009-11-21 19:12:57 |
| i don't like standardization which doesn't allow me to run a program on my hosting shell | 2009-11-21 19:13:27 |
| because that program expects to be properly installed | 2009-11-21 19:13:38 |
| and only root can do that, which i don't have | 2009-11-21 19:13:47 |
*** zapper has joined the channel | 2009-11-21 19:14:12 |
<red> | that is a problem | 2009-11-21 19:14:21 |
<Shelwien> | zapper: you have no patience ;) - http://ps16893.dreamhost.com/cgi-system/chantail.cgi?32 | 2009-11-21 19:14:40 |
<red> | i've had some ..with all that win vista/win7 | 2009-11-21 19:14:56 |
<Shelwien> | yeah | 2009-11-21 19:15:05 |
| unixes have even more of that ;) | 2009-11-21 19:15:30 |
<red> | at least they are democratic | 2009-11-21 19:16:25 |
<Shelwien> | not sure about that | 2009-11-21 19:16:54 |
| user accounts are much more limited there | 2009-11-21 19:17:11 |
| but there's still no actual security anyway | 2009-11-21 19:17:29 |
<red> | you need microsofts certificates else add manifests to your program | 2009-11-21 19:17:37 |
| manifest included might cause some quirks with win xp | 2009-11-21 19:18:04 |
<Shelwien> | well, you need an apple's certificate to even compile an app for iphone ;) | 2009-11-21 19:18:11 |
<red> | at least they have a jailbroken thing going on | 2009-11-21 19:18:35 |
<Shelwien> | sure, like i'd pay them ;) | 2009-11-21 19:19:01 |
<red> | hey .. i was thinking of a gui for single file compressors | 2009-11-21 19:19:54 |
| some one in the forum was talking about them.... | 2009-11-21 19:20:15 |
<Shelwien> | so? there're quite a few already | 2009-11-21 19:20:18 |
| freearc is one, in fact | 2009-11-21 19:20:26 |
<red> | but freearc is also a compressor | 2009-11-21 19:20:41 |
<Shelwien> | not really | 2009-11-21 19:20:47 |
<red> | ? | 2009-11-21 19:20:57 |
<Shelwien> | it does compression, but mostly with external compressors | 2009-11-21 19:21:10 |
| well, the open source ones might be precompiled, dunno | 2009-11-21 19:21:28 |
<red> | open source? | 2009-11-21 19:21:40 |
<Shelwien> | but basically freearc is a haskell shell for single-file compressors | 2009-11-21 19:21:54 |
<red> | bulat has builtin some of the compressors | 2009-11-21 19:22:01 |
<Shelwien> | you can add any to its config | 2009-11-21 19:22:02 |
| and that use it in fa archives | 2009-11-21 19:22:11 |
<red> | yes you can | 2009-11-21 19:22:12 |
| its cumbersome | 2009-11-21 19:22:21 |
| some think tailormade that only provides tar | 2009-11-21 19:22:51 |
| *thing | 2009-11-21 19:22:57 |
<Shelwien> | dunno. i never tested fa actually ;) | 2009-11-21 19:23:32 |
| ...anyway, there still might be a demand for a new GUI like that | 2009-11-21 19:25:28 |
<red> | well from what i gathered ..that newbie compression ethusiast did look into a lot of things | 2009-11-21 19:25:44 |
<Shelwien> | but it has to be fast and light and convenient at that | 2009-11-21 19:26:02 |
| and has to support zip/7z/rar archives to boot | 2009-11-21 19:26:19 |
<red> | peazip ,fa, something named coffee | 2009-11-21 19:26:20 |
<Shelwien> | yeah, though that's too dumb most of the time | 2009-11-21 19:26:51 |
| archiver is a sensitive application | 2009-11-21 19:27:09 |
<red> | i cant find that thread..in the forum now.(.will do lateron) | 2009-11-21 19:27:10 |
<Shelwien> | like, you can't have it failing in some rare cases | 2009-11-21 19:27:39 |
<red> | nah...i'm not thinking of a gui compressor frontend | 2009-11-21 19:27:49 |
<Shelwien> | ;) | 2009-11-21 19:28:04 |
<red> | something like a wrapped for commandline single file compressor | 2009-11-21 19:28:26 |
| *wrapper | 2009-11-21 19:28:32 |
<Shelwien> | good too, i guess | 2009-11-21 19:28:48 |
<red> | that has xml based gui | 2009-11-21 19:28:51 |
<Shelwien> | but you'd still require an archive format and all | 2009-11-21 19:29:12 |
<red> | so a coder and provide a gui for his commandline program | 2009-11-21 19:29:18 |
<Shelwien> | and options to work with that format | 2009-11-21 19:29:23 |
| as to gui btw | 2009-11-21 19:29:43 |
| its also possible to make a web interface ;) | 2009-11-21 19:29:54 |
<red> | all that gui providers is tar or gzip support | 2009-11-21 19:29:54 |
<Shelwien> | like, by opening a web server at some port ;) | 2009-11-21 19:30:23 |
| well, tar is bloated, so its better to forget about it ;) | 2009-11-21 19:30:51 |
<red> | yeah but internet explorer isint nice | 2009-11-21 19:31:05 |
<Shelwien> | well, chrome? ;) | 2009-11-21 19:31:29 |
<red> | and firefox/xulrunner is 30-40 mb | 2009-11-21 19:31:31 |
<Shelwien> | yeah, but you can use any default browser anyway | 2009-11-21 19:31:57 |
<red> | i could | 2009-11-21 19:32:12 |
| but what i want is a GUI that is customizable by the programmer of the compressor | 2009-11-21 19:32:51 |
| according to his needs | 2009-11-21 19:33:05 |
<Shelwien> | well, the main thing imho | 2009-11-21 19:33:17 |
| would be to decide first | 2009-11-21 19:33:26 |
| whats the purpose of this whole app | 2009-11-21 19:33:44 |
<red> | hmm.. | 2009-11-21 19:33:53 |
| its like peazip | 2009-11-21 19:34:06 |
<Shelwien> | like, is it compressing files into a single file for convenience | 2009-11-21 19:34:14 |
| or (remote) backup | 2009-11-21 19:34:19 |
<red> | wait | 2009-11-21 19:34:30 |
<Shelwien> | or effective transfer/storage | 2009-11-21 19:34:39 |
| it requires very different features depending on that | 2009-11-21 19:34:54 |
<red> | WAIT | 2009-11-21 19:35:03 |
* Shelwien waits | 2009-11-21 19:35:13 |
| it does nothing but provides a gui some pretty buttons to click for those(users) who dont want to use commandlines | 2009-11-21 19:36:13 |
<Shelwien> | i understood that, and that's why i'm talking about that ;) | 2009-11-21 19:36:49 |
| i mean, you can't just make a GUI for, say, ccm | 2009-11-21 19:37:06 |
| which would allow you to select any file and compress it | 2009-11-21 19:37:17 |
<red> | the user@ encode forum wanted something so he could pipe the | 2009-11-21 19:37:34 |
<Shelwien> | i mean, how that can be used? | 2009-11-21 19:37:35 |
<red> | progress on screen and provide file browser | 2009-11-21 19:37:51 |
<Shelwien> | sure, but i'm saying that its troublesome | 2009-11-21 19:37:57 |
| also you just reminded me about BWT codecs | 2009-11-21 19:38:07 |
| and BWT codecs are special | 2009-11-21 19:38:30 |
<red> | hmmm... | 2009-11-21 19:38:32 |
<Shelwien> | as they're very blockwise | 2009-11-21 19:38:40 |
| and usually, while it processes a block (commonly quite large) | 2009-11-21 19:39:08 |
| you don't have any progress | 2009-11-21 19:39:13 |
<red> | this is difficult | 2009-11-21 19:39:35 |
<Shelwien> | yeah, i can imagine some alternative approaches though | 2009-11-21 19:40:08 |
<red> | ok then...we'll talk later when i have some code to show for it | 2009-11-21 19:40:10 |
<Shelwien> | ... | 2009-11-21 19:40:20 |
| i'm saying that its not about coding really ;) | 2009-11-21 19:40:31 |
<red> | its not | 2009-11-21 19:40:40 |
<Shelwien> | you just have to know precisely what're you trying to do ;) | 2009-11-21 19:40:45 |
<red> | of course | 2009-11-21 19:40:54 |
<Shelwien> | at least, common use cases | 2009-11-21 19:40:57 |
<red> | indeed i have given some thoughtr | 2009-11-21 19:41:17 |
| shelwin makes a file compression program | 2009-11-21 19:42:12 |
| he can only compress a single file with it | 2009-11-21 19:42:39 |
| his program is commandline(of course) | 2009-11-21 19:43:12 |
| me : i have basic gui of common function a file archiver does | 2009-11-21 19:44:12 |
| i provide an editor(program) for the gui and a way to customize the attibutes | 2009-11-21 19:45:02 |
| now shelwin can distribute his program (compressor) and use my editor to make a GUI for it | 2009-11-21 19:45:56 |
| he has a file archiver | 2009-11-21 19:46:24 |
<Shelwien> | yeah, i got that, but there're still too many questions/options | 2009-11-21 19:46:56 |
<red> | what do you think? | 2009-11-21 19:46:56 |
| hit me... | 2009-11-21 19:47:12 |
<Shelwien> | 1. is GUI supposed to be configured for a single compressor or would be able to support multiple at once? | 2009-11-21 19:47:41 |
<red> | totally up to you | 2009-11-21 19:48:07 |
| at once? | 2009-11-21 19:48:38 |
<Shelwien> | 2. would the compressor interface use pipes, or files; would it process the compressor console messages? or what about an option of hooking the system APIs and/or loading the compressor by yourself? | 2009-11-21 19:49:57 |
| 3. would it only provide options to create and archive and extract it, or something else maybe? (update) | 2009-11-21 19:50:43 |
| 4. filters (precomp?) and combining multiple algorithms (like LZ+ppmd in rar)? | 2009-11-21 19:51:23 |
| 5. SFX? | 2009-11-21 19:51:50 |
<red> | 2. it could have a separate thing(methods) for dealing with messeages | 2009-11-21 19:51:57 |
| no hooking sys api... | 2009-11-21 19:52:13 |
<Shelwien> | why? ;) its convenient | 2009-11-21 19:52:24 |
<red> | 3. only the bare minimum | 2009-11-21 19:52:26 |
| 4 why not | 2009-11-21 19:52:35 |
| 5 could be | 2009-11-21 19:52:44 |
| i'm trying to make this thing to help ....not make a new compressor | 2009-11-21 19:53:30 |
<Shelwien> | yeah, but there's still a question of use cases then | 2009-11-21 19:54:11 |
| what i can imagine could be useful | 2009-11-21 19:54:30 |
| is an archiver GUI for general use | 2009-11-21 19:54:43 |
<red> | least common amongst them | 2009-11-21 19:54:43 |
<Shelwien> | with zip/rar/7z support as i said | 2009-11-21 19:54:53 |
| where advanced users would be able to add existing compressors | 2009-11-21 19:55:30 |
<red> | there will be no support for any compressor | 2009-11-21 19:55:54 |
| from my side | 2009-11-21 19:56:10 |
| anyone can add in whatever they want | 2009-11-21 19:56:24 |
<Shelwien> | well, note that compressor developers won't do that, and won't try to make it compatible (by providing support for using pipes or whatever) | 2009-11-21 19:57:31 |
<red> | addins will be through and xml file or some other easily configurable method | 2009-11-21 19:57:37 |
<Shelwien> | don't you think that compressor developer can just use any IDE to do that instead? | 2009-11-21 19:58:45 |
<red> | well i thought most has trouble with gui and stuff and reinventing the wheel everytime | 2009-11-21 19:59:09 |
<Shelwien> | we do have troubles with gui, yeah, but we don't really reinvent anything | 2009-11-21 19:59:33 |
| nobody needs GUI really | 2009-11-21 19:59:42 |
| (i mean developers) | 2009-11-21 19:59:51 |
<red> | well those with multiple compressors to their name dont | 2009-11-21 19:59:55 |
<Shelwien> | no, i meant something different | 2009-11-21 20:00:17 |
<red> | reinvent....they reuse cod | 2009-11-21 20:00:19 |
| e | 2009-11-21 20:00:22 |
| i mean something diffrent too | 2009-11-21 20:00:36 |
| *meant | 2009-11-21 20:00:42 |
| gui thing by me will mostly be for endusers | 2009-11-21 20:01:30 |
| convenience | 2009-11-21 20:01:44 |
<Shelwien> | yeah, that's my point too | 2009-11-21 20:01:51 |
<red> | but .... | 2009-11-21 20:02:08 |
<Shelwien> | which means that you have to provide ways to handle compressors which weren't specially built for your shell | 2009-11-21 20:02:27 |
<red> | lets see.. i need some incentive for devlopers....hmmm | 2009-11-21 20:03:02 |
<Shelwien> | and i'd repeat the question again: wtf users are supposed to do with that GUI? | 2009-11-21 20:03:21 |
<red> | use it for compression and decompression | 2009-11-21 20:04:12 |
| files/ folders | 2009-11-21 20:04:24 |
<Shelwien> | that's too unspecific | 2009-11-21 20:04:44 |
<red> | well for eg | 2009-11-21 20:04:56 |
| if you add 7zip support | 2009-11-21 20:05:12 |
<Shelwien> | you see, compressing texts, and images, and videos, and dvdisos are all different tasks in fact ;) | 2009-11-21 20:05:26 |
<red> | *that are handled by compressors.... | 2009-11-21 20:06:00 |
<Shelwien> | not quite | 2009-11-21 20:06:16 |
<red> | ? how so | 2009-11-21 20:06:22 |
<Shelwien> | you see, you can't compress dvdisos with paq8 ;) | 2009-11-21 20:06:25 |
| even if you embed it into your gui ;) | 2009-11-21 20:06:37 |
| and most of the usual stuff is precompressed already | 2009-11-21 20:07:16 |
| like music/video/images/program installs | 2009-11-21 20:07:27 |
| so your GUI either has to contain something like precomp in it | 2009-11-21 20:07:44 |
| or whether won't be any differences from zip/rar for practical use | 2009-11-21 20:08:07 |
| *or there | 2009-11-21 20:08:13 |
| ...do you know precomp at all btw? ;) | 2009-11-21 20:08:39 |
<red> | yep | 2009-11-21 20:08:47 |
| nifty little thing | 2009-11-21 20:09:15 |
<Shelwien> | not really, but there're no alternatives atm | 2009-11-21 20:09:32 |
| anyway, my point is that there're some specific use cases for archiver GUIs | 2009-11-21 20:10:25 |
<red> | yes | 2009-11-21 20:10:26 |
<Shelwien> | which might require some work | 2009-11-21 20:10:32 |
<red> | indeed some heavy work | 2009-11-21 20:10:47 |
<Shelwien> | like remote backup (which i'm working on atm btw) | 2009-11-21 20:10:47 |
<red> | multiplexing | 2009-11-21 20:11:16 |
<Shelwien> | or compressed storage with (relatively) random access | 2009-11-21 20:11:34 |
| like resources in games etc | 2009-11-21 20:11:42 |
<red> | i love "pc rips" | 2009-11-21 20:12:00 |
| :) | 2009-11-21 20:12:06 |
<Shelwien> | sure, but these days | 2009-11-21 20:12:18 |
| unfortunately, it can't be done just by using a single compressor | 2009-11-21 20:12:35 |
<red> | yep those bat files | 2009-11-21 20:12:50 |
<Shelwien> | also, don't forget that there're specialized codecs | 2009-11-21 20:13:02 |
| like mp3zip/soundslimmer or optimfrog, or bmf | 2009-11-21 20:13:25 |
<red> | yes | 2009-11-21 20:14:00 |
| ogg is the fav... | 2009-11-21 20:14:14 |
| lossy compression in rips there days | 2009-11-21 20:14:44 |
| *these | 2009-11-21 20:14:49 |
<Shelwien> | whatever, there's no sense to lossily convert all the mp3s to ogg ;) | 2009-11-21 20:14:52 |
| but there're lossless compressors for mp3s ;) | 2009-11-21 20:15:06 |
<red> | i"ll talk to you later | 2009-11-21 20:16:44 |
| need sleep | 2009-11-21 20:17:15 |
<Shelwien> | ;) | 2009-11-21 20:17:37 |
<red> | i"m at encode i'm the one with the turtle .... | 2009-11-21 20:17:59 |
| lets see if you know who i am | 2009-11-21 20:18:12 |
| ..bye | 2009-11-21 20:18:18 |
*** red has left the channel | 2009-11-21 20:18:27 |
<zapper> | i wonder who it could be? | 2009-11-21 20:33:42 |
<Shelwien> | i don't know, but not that i care ;) | 2009-11-21 20:34:07 |
<zapper> | me neither ;) LoL | 2009-11-21 20:34:38 |
<Shelwien> | yeah, i don't know who you are either ;) | 2009-11-21 20:35:14 |
| !grep zapper | 2009-11-21 20:36:15 |
| but you appearing and quitting after a minute was fun anyway ;) | 2009-11-21 20:37:04 |
| you see, its now how irc works ;) | 2009-11-21 20:37:10 |
| when you join, you don't normally see anything that was said before ;) | 2009-11-21 20:37:43 |
<zapper> | as i see it anyone can join this chat as say 'encode' and you never know if it really is encode | 2009-11-21 20:39:36 |
<Shelwien> | in a way | 2009-11-21 20:39:53 |
<zapper> | how? | 2009-11-21 20:40:00 |
<Shelwien> | but its not popular enough for real fakes to appear | 2009-11-21 20:40:09 |
| and then, its possible to register a nick on irc | 2009-11-21 20:40:17 |
| then you would be able to use it only if you know a password | 2009-11-21 20:40:45 |
| ...i don't think its compatible with that webchat though | 2009-11-21 20:41:22 |
<zapper> | i really dont like the way people can use any nick they like her. | 2009-11-21 20:41:45 |
| e | 2009-11-21 20:41:48 |
| can anything be done about this? | 2009-11-21 20:42:17 |
<Shelwien> | you mean, with other people? | 2009-11-21 20:42:36 |
| but on the forum their nicks are not any more comprehensible actually | 2009-11-21 20:43:10 |
| usually you only know who it is from programs they post etc | 2009-11-21 20:43:30 |
<zapper> | but at least they need to sign in at the forum | 2009-11-21 20:44:01 |
| btw i enjoyed your chat with big red | 2009-11-21 20:44:21 |
<Shelwien> | do you know the log url btw? | 2009-11-21 20:45:14 |
| i mean this: http://ps16893.dreamhost.com/log/ | 2009-11-21 20:45:46 |
| you missed some more people lately ;) | 2009-11-21 20:46:11 |
<zapper> | just looking at the log files | 2009-11-21 20:48:01 |
| LoL | 2009-11-21 20:48:04 |
| i dont really like to join in these live chats myself but usually enjoy it when there's some interesting chat in progress | 2009-11-21 20:50:00 |
<Shelwien> | well, its more like "talk to shelwien" than compression channel actually ;) | 2009-11-21 20:51:20 |
<zapper> | do you have to treat these chats as 'topic'/'off topic' like at the forum | 2009-11-21 20:51:29 |
| are the rules strict here? | 2009-11-21 20:51:38 |
<Shelwien> | not really | 2009-11-21 20:51:49 |
| there's some bodybuilding chat here | 2009-11-21 20:52:06 |
| toffer and lasse went about it for hours ;) | 2009-11-21 20:52:16 |
<zapper> | ah toffer's here. this should be an interesting chat | 2009-11-21 20:52:51 |
<Shelwien> | also sami appears occasionally | 2009-11-21 20:53:11 |
<zapper> | that's the way i see this chat | 2009-11-21 20:53:18 |
<Shelwien> | well, it was almost like i'd want to see it when ccm decompiling project was active | 2009-11-21 20:53:56 |
<zapper> | i wish someone else would join the chat so i can sit back and enjoy again :) | 2009-11-21 20:55:31 |
| im not much of a chat person myself | 2009-11-21 20:55:45 |
| as you probably guessed LoL | 2009-11-21 20:55:58 |
<Shelwien> | asmodean and pinc here are like that too | 2009-11-21 20:56:46 |
| but at least they lurk in the channel ;) | 2009-11-21 20:56:55 |
| adding numbers to the crowd ;) | 2009-11-21 20:57:15 |
<zapper> | why do you think the forum has been so quiet lately? | 2009-11-21 20:57:35 |
<Shelwien> | some seasonal thing maybe | 2009-11-21 20:57:57 |
| toffer has his thesis etc | 2009-11-21 20:58:06 |
| many are busy with work too | 2009-11-21 20:58:20 |
<zapper> | maybe? but i don't like it | 2009-11-21 20:58:21 |
<Shelwien> | sure, but i can't really fix it by myself ;) | 2009-11-21 20:58:41 |
<zapper> | nor i | 2009-11-21 20:58:54 |
| :) | 2009-11-21 20:58:59 |
<Shelwien> | you can in fact ;) | 2009-11-21 20:59:06 |
<zapper> | i wish i could :) | 2009-11-21 20:59:18 |
<Shelwien> | i mean, i have 600 posts in that forum as it is | 2009-11-21 20:59:51 |
| so not that i can post much more | 2009-11-21 21:00:27 |
<zapper> | same here | 2009-11-21 21:00:50 |
| the forum seems to get reaaly interesting and busy then it just seems to come to a stop | 2009-11-21 21:01:48 |
| dead stop that is :) | 2009-11-21 21:02:00 |
<Shelwien> | well, the program maintenance threads save it in a way | 2009-11-21 21:02:25 |
<zapper> | no one writing a super powerful compressor lately either LoL | 2009-11-21 21:02:46 |
<Shelwien> | well, there're a few paq8 branches ;) | 2009-11-21 21:03:14 |
<zapper> | yea but... | 2009-11-21 21:03:55 |
<Shelwien> | well, compression became kinda specialized | 2009-11-21 21:04:24 |
| as i tried to explain to "red" here | 2009-11-21 21:04:46 |
| nobody really needs a rar equivalent with different codecs | 2009-11-21 21:05:08 |
<zapper> | i think many members are waiting for the next magic compressor like CCM was at the time | 2009-11-21 21:05:18 |
<Shelwien> | well, i still don't understand what's so special about ccm even now | 2009-11-21 21:05:51 |
| !grep results.txt | 2009-11-21 21:05:59 |
<zapper> | did you say sami sometimes joins this chat? | 2009-11-21 21:06:14 |
<Shelwien> | yeah | 2009-11-21 21:06:19 |
| but i meant this: | 2009-11-21 21:06:58 |
| 32.235s 32.359s ccm 1.30c 5 | 2009-11-21 21:06:59 |
| 28.187s 29.515s 22004883 ccm_sh1d9e | 2009-11-21 21:07:08 |
<zapper> | ccm is considered special because it gives paq like compression but is quite fast. thats the way i understand it. | 2009-11-21 21:07:26 |
<Shelwien> | its not even nearly paq-like | 2009-11-21 21:07:41 |
| well, its only paq-line in the sense that its CM too | 2009-11-21 21:07:54 |
<zapper> | i understand that but its the way its perceived to be paq-like | 2009-11-21 21:08:08 |
<Shelwien> | well, its some weird fashion thing imho | 2009-11-21 21:08:27 |
<zapper> | it basically the way members see it that make it special | 2009-11-21 21:08:41 |
| if people think is junk, then it will be. even if its good. | 2009-11-21 21:09:08 |
| thats the way i undestand it anyway | 2009-11-21 21:09:32 |
<Shelwien> | yeah, kinda, but its more about whether they see it at all | 2009-11-21 21:09:34 |
| like i have a ccm source here and nobody cares ;) | 2009-11-21 21:09:57 |
| and that version is 10% faster even ;) | 2009-11-21 21:10:09 |
<zapper> | i care but people dont want to tread on toffers toes with this | 2009-11-21 21:10:29 |
<Shelwien> | dunno really | 2009-11-21 21:11:04 |
<zapper> | he's a well respected member of the forum | 2009-11-21 21:11:13 |
<Shelwien> | there're quite a lot of different things to do | 2009-11-21 21:11:15 |
<zapper> | is all about the people | 2009-11-21 21:11:49 |
<Shelwien> | well, toffer is basically the only person who discusses compression theory with me ;) | 2009-11-21 21:11:54 |
| but that doesn't mean that you can't make a compressor which would work better than his | 2009-11-21 21:12:34 |
| as he only concentrates on specific aspects of the model | 2009-11-21 21:12:54 |
| ...same with me, i guess | 2009-11-21 21:13:08 |
<zapper> | btw i meant chris not toffer sorry. toffer is not the author of ccm. sorry | 2009-11-21 21:13:11 |
<Shelwien> | toffer's cmm is not that bad either ;) | 2009-11-21 21:13:33 |
<zapper> | true | 2009-11-21 21:15:11 |
| will come back here later. my wife has just asked me to do a little job here :) | 2009-11-21 21:15:56 |
<Shelwien> | ;) | 2009-11-21 21:16:03 |
| !next | 2009-11-21 21:16:05 |