<Skymmer> Let's see...2009-09-01 16:04:54
 Damn, works ;)2009-09-01 16:05:11
 Hi Eugene! And everybody else...2009-09-01 16:05:48
*** pinc has left the channel2009-09-01 16:14:07
 damn2009-09-01 16:33:36
*** Skymmer has left the channel2009-09-01 16:33:40
<sami> here is the checksum, so that osman can join the benchmark too without doing much coding :-) http://pastebin.com/d56204702009-09-01 16:39:48
<Simon|B> what are you guys doing at the moment?2009-09-01 16:48:25
<Shelwien> i'm trying to fix scan5 to comply with sami's conditions ;)2009-09-01 16:49:18
<Simon|B> scan5? I'm out of any information :-D2009-09-01 16:50:16
<Shelwien> apparently we're making a directory traversal benchmark ;)2009-09-01 16:50:41
<Simon|B> ok that's an information I can work with^^2009-09-01 16:52:22
<Shelwien> simon: there was also http://shelwien.googlepages.com/sami_template_v2.rar2009-09-01 16:53:46
 and osman's scanner which he just posted2009-09-01 16:53:55
<Simon|B> didn't notice yet2009-09-01 17:08:26
 toffer?2009-09-01 17:09:00
<Shelwien> ok, filesort works, more or less2009-09-01 17:11:18
 where's the checksum again?..2009-09-01 17:11:31
<sami> http://pastebin.com/d56204702009-09-01 17:11:38
<Shelwien> its took quite some time ;)2009-09-01 17:12:03
 i completely forgot how scan5 works2009-09-01 17:12:11
 and its structure is inconvenient for this kind of sort2009-09-01 17:12:26
 ...that is, if i understood you right and file paths have to be sorted2009-09-01 17:12:57
<sami> yes. "dir1/b.txt" < "dir2/a.txt"2009-09-01 17:13:21
<Shelwien> i skipped subdirs, is that right too?2009-09-01 17:14:10
 i mean, only files with paths, no dirs alone2009-09-01 17:14:28
<sami> yes, only files2009-09-01 17:14:46
<Shelwien> ok, then i'd add crc and it'd be ready2009-09-01 17:14:58
 sami: do you want to compile it yourself?2009-09-01 17:22:00
 sami2009-09-01 17:28:36
 http://shelwien.googlepages.com/src_scan6.rar2009-09-01 17:28:38
 scan.exe is what's intended for benchmark2009-09-01 17:28:50
<sami> ok, no source required2009-09-01 17:29:07
<Shelwien> whatever2009-09-01 17:29:18
 there're both sources and executables2009-09-01 17:29:27
 scan-dump.exe is the same but produces also dump*.txt files2009-09-01 17:29:38
 which can be used to check if crc won't match2009-09-01 17:29:59
*** pinc has joined the channel2009-09-01 18:07:26
<sami> scan.exe crashes after it prints out the memused=... line. it works in smaller directories but with the 250k files this happens2009-09-01 18:10:46
 the number of files it prints is correct2009-09-01 18:12:37
<Shelwien> i suspected something like that ;)2009-09-01 18:20:06
 ok, i'd try to fix it ;)2009-09-01 18:20:14
*** toffer has left the channel2009-09-01 18:25:24
 reuploaded (the same link)2009-09-01 18:25:42
 sami?2009-09-01 18:28:17
<sami> ok, I'll try again soon2009-09-01 18:48:29
*** chornobyl has left the channel2009-09-01 18:55:28
 our checksums don't match, but here are first unofficial results: myscan: 3.1s (kernel: 2.0, user: 1.0), scan6: 3.6 (kernel 1.9, user: 1.6)2009-09-01 19:48:36
 archiver template that has checksum added (with full sort) is: 21s (10s, 10s)2009-09-01 19:52:25
*** pinc|mirror has joined the channel2009-09-01 20:05:19
*** pinc has left the channel2009-09-01 20:10:30
*** pinc|mirror has left the channel2009-09-01 20:18:03
*** toffer has joined the channel2009-09-01 20:37:28
<toffer> hi again2009-09-01 20:37:33
<Simon|B> hi. You got the links toffer?2009-09-01 20:39:30
<toffer> nope2009-09-01 20:41:21
 i had to quit2009-09-01 20:41:27
 could you post these again?2009-09-01 20:41:35
<Simon|B> dunno^^2009-09-01 20:45:16
 http://www.file-upload.net/download-1861346/log.txt.html2009-09-01 20:45:43
 old2009-09-01 20:45:45
 no new^^2009-09-01 20:45:51
 http://www.file-upload.net/download-1861469/logo.txt.html2009-09-01 20:46:05
 old2009-09-01 20:46:07
 for new scroll down to line 14612009-09-01 20:48:59
<toffer> this one doesn't have that much lines2009-09-01 20:51:57
<Simon|B> no but new is the first ;)2009-09-01 20:52:50
 there is some old content first but not so the second2009-09-01 20:53:05
<toffer> that looks pretty good2009-09-01 20:55:14
 i decided to add some kind of date/version number stuff and some information about the difference between minima2009-09-01 20:59:01
<sami> Shelwien, about the improvment to rcs. the sh1d9e? is it about compiler tweaks only or is there something new? what's "__declspec(allocate(SEGNAME))"?2009-09-01 21:03:26
<Simon|B> @ toffer adding where?2009-09-01 21:06:43
<toffer> to the log output2009-09-01 21:08:08
<Simon|B> oh right, it isn't there2009-09-01 21:10:53
 it's in the console so I thought there also2009-09-01 21:11:17
 where is the graph? :-P2009-09-01 21:14:22
<toffer> it'll take some time2009-09-01 21:20:55
 since i'm under windows now2009-09-01 21:21:01
*** Simon|B has left the channel2009-09-01 21:36:05
*** sami has left the channel2009-09-01 21:54:45
 gn8 guys2009-09-02 00:11:47
*** toffer has left the channel2009-09-02 00:12:12
<Shelwien> ...and what's "myscan"?2009-09-02 02:54:08
*** pinc has joined the channel2009-09-02 06:49:59
*** pinc|mirror has joined the channel2009-09-02 06:58:23
*** pinc has left the channel2009-09-02 06:59:01
*** pinc has joined the channel2009-09-02 07:23:56
*** pinc|mirror has left the channel2009-09-02 07:29:28
*** pinc has left the channel2009-09-02 07:34:05
*** pinc has joined the channel2009-09-02 08:01:29
*** toffer has joined the channel2009-09-02 08:55:15
<toffer> hi2009-09-02 08:55:18
<Shelwien> hi2009-09-02 08:55:59
*** Simon|B has joined the channel2009-09-02 09:16:22
*** Simon|B has left the channel2009-09-02 09:20:47
*** Simon|B has joined the channel2009-09-02 10:37:44
<osman> hi2009-09-02 11:05:02
 is there any way to get unicode arguments from linux command line?2009-09-02 11:05:18
 i mean what's the equalivent way of "argv" for unicode chars2009-09-02 11:05:51
<Shelwien> ;)2009-09-02 11:21:36
 i think they work with utf8 ;)2009-09-02 11:21:53
 http://eyegene.ophthy.med.umich.edu/unicode/2009-09-02 11:23:30
 btw, osman2009-09-02 11:43:16
 "myscan" is your version or what? ;)2009-09-02 11:43:23
 i mean2009-09-02 11:43:48
 <sami> our checksums don't match, but here are first unofficial results: myscan: 3.1s (kernel: 2.0, user: 1.0), scan6: 3.6 (kernel 1.9, user: 1.6)2009-09-02 11:43:48
 <sami> archiver template that has checksum added (with full sort) is: 21s (10s, 10s)2009-09-02 11:43:48
<Simon|B> What shall be the use of this archiver template? Will this be used by NZ and Bit?2009-09-02 12:01:24
<Shelwien> sami's archiver template is used by flashzip2009-09-02 12:02:58
 and me and osman written our own implementations2009-09-02 12:03:27
 which parse the directory tree much faster2009-09-02 12:03:44
 and then2009-09-02 12:04:10
 it seems like a good simple task2009-09-02 12:04:18
 to compare various languages and programming styles2009-09-02 12:04:41
 (like Sami's version is mainly slow because of STL)2009-09-02 12:04:41
 so we're trying to make a benchmark2009-09-02 12:04:52
 ...2009-09-02 12:05:14
 and then, this code is also necessary for any program which works with multiple files2009-09-02 12:05:42
 like the backup tool which i'm supposed to write now ;)2009-09-02 12:06:12
<Simon|B> yeah such tasks are something I wanted to know ;)2009-09-02 12:15:17
 sami uses his template?2009-09-02 12:15:26
<Shelwien> no ;)2009-09-02 12:15:36
 he written it after nz2009-09-02 12:15:43
 and nz's version is twice faster ;)2009-09-02 12:15:55
<Simon|B> funny :-D2009-09-02 12:16:09
<osman> @Shelwien: i'm very busy to learn those craps about "platform compatibility" :) i really don't know about "myscan". i've packed my version as "bit"2009-09-02 12:26:45
<Shelwien> weird2009-09-02 12:27:15
 maybe some new version of sami's?..\2009-09-02 12:27:23
<osman> maybe a kind of ripped version of nz2009-09-02 12:27:35
 actually it's unfair. because both of us supporting unicode2009-09-02 12:28:02
<Shelwien> that's unlikely imho2009-09-02 12:28:02
 afaik nz version was twice faster than archiver template2009-09-02 12:28:27
 which is still much slower than scan5 ;)2009-09-02 12:28:35
 maybe he'd written a new one after seeing scan5 or something ;)2009-09-02 12:28:56
*** Shelwien has left the channel2009-09-02 12:29:17
*** Guest9968193 has joined the channel2009-09-02 12:29:21
<osman> how are you "ghost" ? :)2009-09-02 12:29:43
<Shelwien> read /ns help ghost2009-09-02 12:29:56
 you have to be registered though2009-09-02 12:30:05
<osman> can't use any command in web client2009-09-02 12:30:11
<Shelwien> you can2009-09-02 12:30:20
 send a PM to nickserv2009-09-02 12:30:36
 that /ns is a shortcut to /msg nickserv2009-09-02 12:30:54
<osman> thanks for the explanation2009-09-02 12:31:16
 btw, on linux you are right2009-09-02 12:31:27
 all command lines argument passed as utf-82009-09-02 12:31:37
<Shelwien> yeah2009-09-02 12:31:40
 simplifies things though2009-09-02 12:31:50
 damned MS ;)2009-09-02 12:31:58
<osman> and also i found that "implementing posix api supported code" is much easier to "find a compatible version of standard I/O (fopen etc) for posix"2009-09-02 12:33:08
 btw, afair you have a mac in there right?2009-09-02 12:33:58
 macbook or mac-mini (which is my favorite :] )?2009-09-02 12:34:22
<Shelwien> macbook2009-09-02 12:34:45
 ...with winXP on it...2009-09-02 12:35:00
<osman> i wonder what's the operation system? darwin? carbon?2009-09-02 12:35:01
 grrr...then why do you use it!? :)2009-09-02 12:35:25
<Shelwien> its small and light2009-09-02 12:35:48
 and relatively good display matrix2009-09-02 12:35:57
 and good accumulators2009-09-02 12:36:03
 and core22009-09-02 12:36:04
 ...but i hate its touchpad and keyboard, really2009-09-02 12:36:19
<osman> at a time, i have tried to install macosx tiger to wmvare but no succeeded :)2009-09-02 12:36:41
<Shelwien> but with an external mouse and keyboard... and display... its a nice working computer ;)2009-09-02 12:36:47
<osman> i've used several mac os for a while (~3-6 months)2009-09-02 12:37:12
 then you should try mac-mini :)2009-09-02 12:37:27
<Shelwien> http://thepiratebay.org/torrent/4707572/MacOS_10.5.5_VMWARE_image2009-09-02 12:37:28
<osman> do you have "partnership" with piratebay? :)2009-09-02 12:37:54
<Shelwien> i had to write a tutorial on it recently2009-09-02 12:37:54
<osman> for what?2009-09-02 12:38:17
<Shelwien> for other developers to test the codec2009-09-02 12:38:20
<osman> installing macs? :)2009-09-02 12:38:21
 which codec?2009-09-02 12:38:29
<Shelwien> the audio codec2009-09-02 12:38:43
<osman> audio?2009-09-02 12:38:44
 ah..ok2009-09-02 12:38:48
<Shelwien> which i'm developing2009-09-02 12:38:51
<osman> what's the current development status of it?2009-09-02 12:38:59
<Shelwien> we have to port it to mac2009-09-02 12:39:00
 dunno2009-09-02 12:39:09
 its usable for years2009-09-02 12:39:25
 but we keep working on it2009-09-02 12:39:30
 and occasionally start from scratch2009-09-02 12:39:39
<osman> making a iTunes plugin will be very good2009-09-02 12:39:43
<Shelwien> somehow they're making a separate macos app for it2009-09-02 12:40:14
 and i use console encoder and directshow filters2009-09-02 12:40:32
 (which i written)2009-09-02 12:40:36
<osman> btw, what's internal detail of the codec? i mean lossless, lossy?2009-09-02 12:40:39
<Shelwien> i also made a plugin for XBMC2009-09-02 12:40:45
 lossy2009-09-02 12:40:52
<osman> also LPC, DCT, wavelet?2009-09-02 12:40:52
<Shelwien> MDCT2009-09-02 12:41:00
 with relatively large window though2009-09-02 12:41:20
 10242009-09-02 12:41:23
<osman> mp3s and ogg are typical MDCT codecs. so, what's the main advantage of this codec? (don't think i'm in offensive way. just wondering)2009-09-02 12:41:59
<Shelwien> whatever, like i care2009-09-02 12:42:14
 it has better quality at the same bitrate though2009-09-02 12:42:35
 we use it at ~70kbps now2009-09-02 12:42:50
 and its supposed to be on par with HE-AAC 96kbps2009-09-02 12:43:06
<osman> that's really good then :)2009-09-02 12:43:12
<Shelwien> you can do a listening test btw2009-09-02 12:43:32
 we had a large-scale one recently2009-09-02 12:43:56
<osman> :) i would compare them by listening + plus seeing residues + PSNR value for whole audio :)2009-09-02 12:44:12
<Shelwien> however it takes like 2 hours so its unlikely that you'd do that2009-09-02 12:44:35
 and spectral analysis / PSNR are irrelevant2009-09-02 12:45:03
 unless you'd provide similar results for other codecs2009-09-02 12:45:23
<osman> i think, this kind of tests are really required for lossy compressor (both audio and image/video)2009-09-02 12:45:35
 i mean comparing residues and PSNR with original source+mp3 and original source+lamix (or whatever)2009-09-02 12:46:20
<Shelwien> as I said, who cares about PSNR at low bitrates2009-09-02 12:46:47
 its subjective quality that matters2009-09-02 12:46:58
 anyway, i can give you a console encoder/decoder2009-09-02 12:47:55
 if you're so interested ;)2009-09-02 12:48:06
<osman> just for seeing equelivant bitrates at the other codes (some people really like that kind of comparations)2009-09-02 12:48:16
 maybe later :)2009-09-02 12:48:28
<Shelwien> well, i don't care either way2009-09-02 12:48:41
 i'm only doing entropy coding and system stuff there2009-09-02 12:48:56
 also container format etc2009-09-02 12:49:06
 and i'm not a least bit happy about their psychoacoustic model2009-09-02 12:49:19
<osman> it's bad you mean?2009-09-02 12:49:46
<Shelwien> not really2009-09-02 12:49:52
 at least i did that LT39 myself2009-09-02 12:50:02
 and barely heard any difference with originals2009-09-02 12:50:24
 and believe me, i really tried2009-09-02 12:50:36
 just that its mainly manually configured, not quite adaptive2009-09-02 12:51:53
 (i mean, for different bitrates etc)2009-09-02 12:52:00
 and programming quality is really bad2009-09-02 12:52:23
<Simon|B> Isn't VirtualBox better then VmWare now?2009-09-02 13:54:03
 Anyone used it?2009-09-02 13:54:13
<Shelwien> i used VPC2009-09-02 13:59:11
 liked it more somehow2009-09-02 13:59:19
 more user-friendly somehow2009-09-02 13:59:26
 now have to use vmware though2009-09-02 14:00:24
 ...killing off all its services is really annoying2009-09-02 14:00:48
<Simon|B> With vmware it's often difficult to do easy tasks2009-09-02 14:01:30
 don't mention many problems it has2009-09-02 14:01:57
 the I will try virtualbox :)2009-09-02 14:02:07
 An OSX machine is inteesting, also that I have a native system here :-D2009-09-02 14:04:17
*** Simon|B2 has joined the channel2009-09-02 14:08:04
<Simon|B2> No 10.6 VM yet :(2009-09-02 14:09:31
*** Simon|B has left the channel2009-09-02 14:10:26
*** toffer has left the channel2009-09-02 14:11:15
*** toffer has joined the channel2009-09-02 14:18:34
*** Shelwien has left the channel2009-09-02 14:28:53
*** Shelwien has joined the channel2009-09-02 14:30:58
 How big/how much space does the posted vmware have @ Shelwien?2009-09-02 14:40:23
<Shelwien> 8Gb or something i think2009-09-02 14:50:13
 i barely installed xcode and intelc there2009-09-02 14:50:25
 but i think there should be a way to resize it2009-09-02 14:50:48
<Simon|B2> I remeber that this isn't possible2009-09-02 14:51:08
<Shelwien> well, adding another partition surely is2009-09-02 14:51:30
 and imho resize should be possible too2009-09-02 14:52:05
<Simon|B2> http://www.leonmeijer.nl/archive/2007/05/07/25.aspx2009-09-02 14:52:09
 yes it is possible2009-09-02 14:52:15
<Shelwien> at least i was able to cut off 60G from my partition to install windows ;)2009-09-02 14:52:22
<Simon|B2> sure you can set it at the beginning but I remembered that you can't change it later - wrong as noticed now :-D2009-09-02 14:53:19
*** sami has joined the channel2009-09-02 15:40:56
<sami> hi!2009-09-02 15:41:14
 http://compressionratings.com/s_scan.html2009-09-02 15:41:19
<Shelwien> hi2009-09-02 15:41:24
<sami> the checksums match now for both of our programs2009-09-02 15:42:24
 but the numbers I quoted yesterday must have been wrong, I suspect, my program didn't do any sorting then2009-09-02 15:42:55
<Shelwien> ok2009-09-02 15:43:01
 i was going to replace the sorting though ;)2009-09-02 15:43:13
<sami> the 0w version means wide char version2009-09-02 15:43:16
<Shelwien> the one used there is qsort+strcmp2009-09-02 15:43:30
 on full paths too2009-09-02 15:43:39
 i kinda think that its a wrong constraint for this task actually2009-09-02 15:44:09
 as my implementation is perfectly ok for an archiver2009-09-02 15:44:19
 but its not easy to produce a sorted path list from it2009-09-02 15:44:33
<sami> "sscan v0 I" is supposed to be put into nz but it appears I'm too lazy to do that. the sscan was made after you sent me the benchmarks of archiver template2009-09-02 15:46:33
<Shelwien> ;)2009-09-02 15:46:59
 well, its relevant for me now2009-09-02 15:47:11
 as apparently i'm really writing a remote backup utility... aka archiver for conexware ;)2009-09-02 15:47:37
 so i'd probably have to rewrite that scan too2009-09-02 15:48:02
 i've got an idea to use utf8 for name storage etc2009-09-02 15:48:14
<sami> do I recall osman said his bitscan is faster than archiver template? doesn't seem so here. the one listed here I meant to put online too but forgot and now I already shut down the benchmark machine2009-09-02 15:48:38
 do I recall -> do I recall correctly2009-09-02 15:48:53
<Shelwien> it probably doesn't scale well for that many file2009-09-02 15:48:57
 *files2009-09-02 15:49:01
<sami> ok2009-09-02 15:49:07
<Shelwien> so, is there any sense to update scan6 with some STL sorting?2009-09-02 15:49:38
 and I'd appreciate if you could give me a good STL line for sorting char* array ;)2009-09-02 15:50:47
<sami> well, if you can observe an improvment, but I doubt it2009-09-02 15:51:17
<Shelwien> i think that my current implementation is very inefficient for files with paths2009-09-02 15:51:45
 like, strcmp might even call strlen from it2009-09-02 15:52:17
 and i know the lengths...2009-09-02 15:52:22
 btw, what are these ISLV?2009-09-02 15:53:46
<sami> sscan uses string class, so comparison is done with memcmp() actually, or wmemcmp for 0w2009-09-02 15:53:48
<Shelwien> well, it might be still better than strcmp2009-09-02 15:54:11
<sami> "I" is my own linked list, "S" is std::set, "L" is std::list, "V" is std::vector2009-09-02 15:54:33
<osman> @shelwien: seems you are right. seems utf8 coded storage is really needed (especially in my case). because most of platforms supports utf8 by default. only exception seems windows2009-09-02 15:54:39
<sami> "I" uses my own sorting naturally2009-09-02 15:54:56
<Shelwien> yeah, and there's utf322009-09-02 15:54:57
<sami> btw. how to convert wchar_t to utf8 in windows?2009-09-02 15:55:32
<Shelwien> manually? ;)2009-09-02 15:56:03
<osman> yep. manually ;)2009-09-02 15:56:11
<Shelwien> i think that's the best solution anyway ;)2009-09-02 15:56:18
<osman> utf32 is actually very abstract layer. and we actually need 21-bits representation ;)2009-09-02 15:56:39
<sami> :-( I would really need that because I plan utf8 for nz windows. currently it works fine in utf8 linux :-) since no changes are needed2009-09-02 15:56:50
<Shelwien> still >162009-09-02 15:56:51
<osman> also there is a "standard" implementation about unicode compression ;)2009-09-02 15:56:54
 "still >16" ?2009-09-02 15:57:17
 all of unicode codes are reversible. so, what's the problem?2009-09-02 15:57:53
<Shelwien> utf8>utf162009-09-02 15:58:09
<osman> and also, i may even use utf8 strings internally. because there are several special codes to define multi-byte characters2009-09-02 15:58:32
<Shelwien> "Round-trip conversion is easy between UTF-8 and UTF-16. To translate to UTF-8 and vice versa in Win32 programming, set the MultiByteToWideChar and WideCharToMultiByte codepage parameter to CP_UTF8."2009-09-02 15:58:32
 so there is an official way to do that2009-09-02 15:58:52
 but you can imagine the overhead ;)2009-09-02 15:58:59
 http://msdn.microsoft.com/en-us/goglobal/bb978451.aspx2009-09-02 15:59:35
<osman> in a real archiver, string overhead would not be so much. if we would talk about xml parser, interpreter etc. that can be worthy imo2009-09-02 15:59:57
<Shelwien> just imagine something like sami's test case2009-09-02 16:00:25
 with 250k files2009-09-02 16:00:30
 for every one of which you'd have to call WideCharToMultiByte()2009-09-02 16:00:51
<osman> %) you're right2009-09-02 16:00:52
<Shelwien> maybe two times2009-09-02 16:00:55
 (as there're name shortcuts)2009-09-02 16:01:14
<osman> but, what do you think for real usages. i mean only considering extensions on sorting2009-09-02 16:01:29
<Shelwien> well, i had that idea to use my own bitcoding instead of utf8 ;)2009-09-02 16:02:04
<osman> huffman like codes ;)2009-09-02 16:02:22
<Shelwien> optimal ordered more likely2009-09-02 16:02:37
 as sorting is required2009-09-02 16:02:43
 though in this case actually2009-09-02 16:02:51
 there're separate strings to sort2009-09-02 16:02:57
 so name rangecodes can be used too ;)2009-09-02 16:03:06
*** pinc has left the channel2009-09-02 16:03:14
<osman> btw, did you realized that wchar_t actually is a variable coding on windows? %)2009-09-02 16:03:37
<Shelwien> no2009-09-02 16:03:48
<osman> it can represent >0xFFFF limit2009-09-02 16:03:58
<Shelwien> dunno2009-09-02 16:04:08
 apparently they used these headers2009-09-02 16:04:16
 in some weird environments2009-09-02 16:04:24
<osman> so, that's why i began to think use of a "universal" representation2009-09-02 16:04:33
<Shelwien> as there're macro checks for MacOS occasionally etc2009-09-02 16:04:39
 in like win*.h files %)2009-09-02 16:04:46
<osman> i mean you can map any utf-32 character (of course exception with non-defined code points) to utf-16 or utf-8. and both utf-8 and utf-16 have a special ranges in bytes which is called as surrogates. surrogates can be interpreted as control words which can be used to decode multibyte sequence.2009-09-02 16:07:00
 so, in short, in windows a single wchar_t DOES NOT ALWAYS reflect a unicode codepoint2009-09-02 16:08:07
<Shelwien> ah, that2009-09-02 16:08:54
 ok, i forgot ;)2009-09-02 16:09:08
<osman> so, even a simple sort method is not valid for utf-16 coded strings2009-09-02 16:09:35
<Shelwien> isn't it valid for utf8?2009-09-02 16:10:02
 why not for utf16 then?2009-09-02 16:10:07
<osman> i'm not sure they are valid (for both utf-8 and utf-16)2009-09-02 16:10:28
 high bits are used as surrogates btw. so, maybe it can be valid. i should look them carefully2009-09-02 16:10:59
 seems i'll use utf-8 at the end :)2009-09-02 16:11:36
<Shelwien> same here probably2009-09-02 16:11:57
 btw did you see the sami's scanner benchmark?2009-09-02 16:12:17
<osman> even for internal usage too. because utf-32 in linux is too much for me. which requires 4x memory2009-09-02 16:12:24
 yep. i briefly looked it2009-09-02 16:12:32
<Shelwien> of course2009-09-02 16:12:36
<osman> btw, did i mention about utf-16 endian problem? :)2009-09-02 16:13:16
<Shelwien> ;)2009-09-02 16:13:27
<osman> so, for me there is 2 choice: utf-8 and utf-322009-09-02 16:13:34
 and i know which one is really better :)2009-09-02 16:13:42
<Shelwien> fortunately endian problems are unlikely in windows ;)2009-09-02 16:13:50
<osman> yep. but, i decided to write BIT for both unix and windows platforms in parallel. so, i could find a universal solutions for both of them.2009-09-02 16:14:34
 i've looked lazarus which supports almost all platforms2009-09-02 16:15:08
 it has full unicode support for all platform except GTK12009-09-02 16:15:26
 and it uses utf-8 internally :)2009-09-02 16:15:37
 seems even on windows2009-09-02 16:15:45
<Shelwien> well, why not2009-09-02 16:16:02
 if windows has native support for it too2009-09-02 16:16:14
<sami> so how can conexware afford to develop remove backup util from scratch with fully original archiver/compression?2009-09-02 16:17:03
 to me that just doesn't compute2009-09-02 16:17:45
<osman> windows support UCS-2 in old days (which is limited 0x0000-0xFFFF in ranges), but with winNT4, winXP, win2000 and higher, windows supports variable utf-16 coding in it's apis2009-09-02 16:17:53
<Shelwien> sami: i guess i'm cheap ;)2009-09-02 16:18:58
<osman> lol2009-09-02 16:19:07
 btw, sami what is you real job?2009-09-02 16:21:42
<sami> Shelwien, ok, but still. I'm sure you ask for at least the minimum wage?2009-09-02 16:23:33
<Shelwien> $15/hour2009-09-02 16:23:42
<sami> with my own compression freelance work I'm always faced with a dilemma: should I use my own code that alerady exist for the task. I mean, often they want to own the code, so using my own code would transfer the authorship2009-09-02 16:25:27
<Shelwien> yeah2009-09-02 16:25:48
 fortunately conexware doesn't care about that2009-09-02 16:26:00
 and my main job does2009-09-02 16:26:06
 but they're so "smart" that they couldn't actually patent a single thing2009-09-02 16:26:28
 err...2009-09-02 16:26:34
 i mean, they do get patents2009-09-02 16:26:40
<sami> what has been your solution? I mean suppose conexware requires something that yo uhave already made in your freetime, how do you go about making that, do you sell it or do you "pretend" to be working on it :-) or what do you do?2009-09-02 16:26:49
<Shelwien> but stuff described in there is unrelated to my actual algorithms2009-09-02 16:26:56
 usually neither in fact2009-09-02 16:27:35
 i just use it and don't count the time2009-09-02 16:27:42
 though that depends2009-09-02 16:27:49
 for example, i made dllmerge kinda for my own fun2009-09-02 16:28:04
 but it collided with dejpg job and was useful for it2009-09-02 16:28:20
 so i "sold" it ;)2009-09-02 16:28:23
<sami> when I have been asked prices for the work that I've already done, it's difficult to set price it for them since often there is plenty of original research that took 10-100 time that it would take if I would just code it from scratch (knowing what to do)2009-09-02 16:28:33
<Shelwien> i just count the time2009-09-02 16:28:51
 i'm writing a log (you can see an example in some ccm_sh versions)2009-09-02 16:29:23
 and then bill for the time written in it2009-09-02 16:29:40
 and its been ok for me until now2009-09-02 16:30:03
 there's no sense in requesting a lot anyway2009-09-02 16:30:51
 if you're expensive, you don't keep the job2009-09-02 16:31:14
 they take what they really need and then say bye2009-09-02 16:31:41
 so i just bill for the time i really spend - and that much money that i can get with other job(s)2009-09-02 16:32:34
<sami> have you been required to sign something that you cannot "sell" the dllmerge again?2009-09-02 16:32:45
 I mean that they own it totally?2009-09-02 16:32:54
<Shelwien> not with conexware2009-09-02 16:32:55
<sami> you've got lucky2009-09-02 16:33:34
<Shelwien> they're are quite ok actually even with that new archiver being "free"2009-09-02 16:33:38
 (console version)2009-09-02 16:33:46
 in fact i think i can even press them to open the source2009-09-02 16:34:35
 well, that's reasonable in fact2009-09-02 16:34:51
 patents cost too much2009-09-02 16:35:01
 and there's not much sense in them anyway, if you're not a corporation2009-09-02 16:35:37
<sami> I'm a bit paranoid. for example. one time I was required to build a dll compressor for custom data and not required to provide source code. and they said I could use whatever previously existing original stuff I've got in that project. but I didn't do that because I thought there might be some loophole that they can legally hijack the ownership later and force me to provide the source2009-09-02 16:36:40
<Shelwien> btw, that's the mp3zip patent - http://brevets-patents.ic.gc.ca/opic-cipo/cpd/eng/patent/2467466/summary.html2009-09-02 16:36:49
 really funny imho ;)2009-09-02 16:36:53
 well, i don't think i ever wrote a program which could be that precious to me2009-09-02 16:38:14
 i mean, it might be a lot of work2009-09-02 16:38:35
 but i always have better ideas than i already implemented2009-09-02 16:38:48
 and i have all the previous results as additional bonus2009-09-02 16:39:08
 (tools etc)2009-09-02 16:39:14
 so i'm ok with selling it in whole, source and all, as long as they pay for my time2009-09-02 16:39:57
 also, my main drawback is that i never "work hard" ;)2009-09-02 16:41:22
<sami> how much time (how many hours) have you estimated that you need for conexware for the archiver2009-09-02 16:41:47
<Shelwien> dunno, years probably, in total2009-09-02 16:42:07
 but fortunately there're stages2009-09-02 16:42:17
<sami> so conexware has not required any deadlines?2009-09-02 16:42:24
 right2009-09-02 16:42:40
<Shelwien> and they have customers interested in some algorithms (like remote backup) which are relatively simple to implement, and necessary for a compressor anyway2009-09-02 16:43:21
 they did of course... and i've broken it multiple times ;)2009-09-02 16:43:46
 and fortunately there're not many people2009-09-02 16:43:57
 (i think 3 beside me actually ;)2009-09-02 16:44:10
<sami> even if they pay you 15$/h for 1-2 years that's an investment, I'm surprised that conexware has the cash. I mean how does power archiver sell? I mean it's shareware right? how can that make any profits2009-09-02 16:45:04
<Shelwien> from corporations mainly2009-09-02 16:45:29
 they have to setup some stuff for such tasks on their computers2009-09-02 16:45:52
 and occasionally they get inspections2009-09-02 16:45:59
<sami> how come they have sold it to corporations? I mean which corporations would have use for it?2009-09-02 16:46:00
<Shelwien> so they buy their stuff ;)2009-09-02 16:46:05
 corporations that use computers with windows in them2009-09-02 16:46:25
 users of these computers need some software to unpack zips ;)2009-09-02 16:46:48
 as apparently they're too dumb to use explorer context menus or something ;)2009-09-02 16:47:10
 (though actually PA supports a lot of formats, and zip extensions are the main reason why its used)2009-09-02 16:47:42
 ...2009-09-02 16:47:56
 also, in fact PA is used in some of adobe products ;)2009-09-02 16:48:13
 well, a custom PA-based installer2009-09-02 16:48:23
 ...so i guess, customers are what's important, not the software quality ;)2009-09-02 16:49:14
<sami> :-)2009-09-02 16:50:01
<Shelwien> anyway, it took me ~200 hours overall to write a working dejpg library2009-09-02 16:50:51
 and they paid for it2009-09-02 16:50:58
<Simon|B2> There is mp3 compression availible fopr zip??2009-09-02 16:51:02
<Shelwien> no, only stuffit2009-09-02 16:51:09
 but jpeg recompression is already supported by winzip... and PA ;)2009-09-02 16:51:33
<sami> I can respect conexware for the business then. don't know how they get those deals. I mean how must that thing work practically? they email adobe and ask that would you like an installer or does adobe find them?2009-09-02 16:51:41
<Simon|B2> yeah stuffit recompresses almost everything :-D2009-09-02 16:51:50
 jpeg yes I know2009-09-02 16:51:55
<Shelwien> well, apparently people mostly want to work with zip archives and usable GUI2009-09-02 16:52:58
 so an archiver with zip support has to be installed on each PC2009-09-02 16:53:20
<Simon|B2> They seems to have some (many) custoerms, but why don't they work on a own compression format?2009-09-02 16:53:22
<Shelwien> and corporations have to pay for programs which they install, at least before an occasional inspection ;)2009-09-02 16:54:01
 simon: well, now they've got me, and you can say that they work on it ;)2009-09-02 16:54:30
 i guess, the main problem2009-09-02 16:54:45
 is that small improvements just break the compatibility2009-09-02 16:55:05
 and nobody wants them2009-09-02 16:55:14
 and for some major breakthrough2009-09-02 16:55:30
 you can't just hire a random freelancer2009-09-02 16:55:53
 anyway, PA is a delphi GUI + proprietary 3rd-party libraries (like ziptv)2009-09-02 16:57:37
 they didn't really invest that much in developement... until recently, at least ;)2009-09-02 16:58:02
<Simon|B2> Yeah sure. I mean an own new format. Something like buying NZ :D2009-09-02 16:59:59
<Shelwien> you see, its not that much demand for it2009-09-02 17:00:51
 *there's 2009-09-02 17:01:14
 basically, most people only use zip2009-09-02 17:01:23
 even, most people _unpack_ zip2009-09-02 17:01:42
<Simon|B2> yes this could be.2009-09-02 17:02:11
<Shelwien> and then, there're sometimes some specific related tasks2009-09-02 17:02:24
 like this backup thing2009-09-02 17:02:30
 but you can't use an existing solution for them2009-09-02 17:02:53
 basically, people want a flashy GUI (not even necessary a good one)2009-09-02 17:04:23
 and some interface features2009-09-02 17:04:46
 like ftp client and cd burner in PA ;)2009-09-02 17:05:10
 or, to be more archiver-specific, volumes and error recovery2009-09-02 17:05:27
 and they don't really care about compression2009-09-02 17:05:53
 like, i know this guy, a very professional engineer2009-09-02 17:06:33
 and he said once that there's no sense in my compression improvements2009-09-02 17:07:12
<Simon|B2> yeah I have to have a look if I ever will be close to a release of my own archiver :-D2009-09-02 17:07:20
<Shelwien> if i can't compress the file twice better than "standard" stuff ;)2009-09-02 17:07:29
 and you know, without recompression support for a lot of formats2009-09-02 17:09:00
 its really hard to compete with winrar... or the new winzip which uses LZMA/PPMD/jpeg recompression2009-09-02 17:09:23
<Simon|B2> Sure. That's true, but Winzip also reacted this seems like it was necessary2009-09-02 17:12:14
<Shelwien> well, with pressure from rar/7z it is, probably2009-09-02 17:14:49
*** Skymmer has joined the channel2009-09-02 17:38:34
<sami> Shelwien by the way if you suspect your strcmp is to blame, you can compare it to sscan ilsv (lowercase), sscan doesn't do any sorting then2009-09-02 17:43:27
<osman> a question again :) how can i get rid of "cannot execute binary file" under linux?2009-09-02 18:00:12
<Shelwien> ?2009-09-02 18:06:06
 is it +x?2009-09-02 18:06:12
<toffer> chmod +x2009-09-02 18:11:24
<osman> it does not work2009-09-02 18:11:39
<Shelwien> then its in wrong format2009-09-02 18:12:00
<osman> i'm trying to compile "hello world" like program under cygwin2009-09-02 18:12:05
 with these commands:2009-09-02 18:12:13
 g++-linux bit.cpp -o bit2009-09-02 18:12:23
 ./bit2009-09-02 18:12:25
 does not work last line2009-09-02 18:12:37
<Shelwien> well, that's some weird kind of linux probably2009-09-02 18:12:40
 try adding -static maybe2009-09-02 18:12:55
<osman> $ file bit2009-09-02 18:13:14
 bit: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.42009-09-02 18:13:14
 .3, dynamically linked (uses shared libs), not stripped2009-09-02 18:13:21
<Shelwien> as i said...2009-09-02 18:13:28
 if its ELF doesn't mean that it would run on any linux2009-09-02 18:13:46
<osman> seems compiler did not like "-static" argument ;)2009-09-02 18:14:22
 dozens of messages listed2009-09-02 18:14:37
<Shelwien> well, read the man about how to force it to compile a static binary then2009-09-02 18:14:51
 without these shared libraries2009-09-02 18:14:56
 which your linux might not want to share2009-09-02 18:15:12
<toffer> @osman: if it's cpp add -lstdc++2009-09-02 18:15:52
*** Skymmer has left the channel2009-09-02 18:17:48
<Shelwien> toffer: he's trying to cross-compiler a linux binary under windows2009-09-02 18:17:50
<osman> $ g++-linux bit.cpp -o -lstdc++ bit2009-09-02 18:17:52
<Shelwien> *compile2009-09-02 18:18:11
<osman> collect2: ld terminated with signal 11 [Segmentation fault], core dumped2009-09-02 18:18:13
 obably corrupted stack)2009-09-02 18:18:13
  5 [main] ld 7608 _cygtls::handle_exceptions: Error while dumping state (pr2009-09-02 18:18:16
 bit(.text+0x2bc): In function `Unix::Console::PutChar(char)':2009-09-02 18:18:18
 i got same errors with "-static"2009-09-02 18:18:38
<toffer> didn't kne2009-09-02 18:18:42
 w2009-09-02 18:18:43
*** Skymmer has joined the channel2009-09-02 18:18:44
<Shelwien> its probably a wrong build 2009-09-02 18:19:49
<toffer> so it's built under windows and supposed to run under linux... what about the build envorinment?2009-09-02 18:20:27
<Shelwien> its a cygwin cross gcc build2009-09-02 18:20:51
 but its probably old and broken2009-09-02 18:20:58
<osman> http://metamod-p.sourceforge.net/cross-compiling.on.windows.for.linux.html2009-09-02 18:21:22
 i've followed these article2009-09-02 18:21:39
 *this2009-09-02 18:21:45
<Shelwien> did you compile gcc and binutils?2009-09-02 18:21:53
<osman> ? no2009-09-02 18:22:00
<Shelwien> then its broken ;)2009-09-02 18:22:12
<toffer> ^^2009-09-02 18:22:27
<osman> i've downloaded provided package and installed it2009-09-02 18:22:37
<Shelwien> linuxes are not binary compatible2009-09-02 18:23:03
<toffer> it still doesn't need to be broken2009-09-02 18:23:11
 where did you try to run it?2009-09-02 18:23:21
<osman> in cygwin bash2009-09-02 18:23:31
<toffer> well libraries should be compiled with the same versions2009-09-02 18:23:33
<Shelwien> huh?2009-09-02 18:23:52
 a linux binary on windows?2009-09-02 18:23:58
<toffer> i mean the libraries you link against2009-09-02 18:24:08
<Shelwien> no wonder either ;)2009-09-02 18:24:08
<osman> cygwin comes with it's own bash2009-09-02 18:24:20
<toffer> it's not about the shell i guess2009-09-02 18:24:33
 i had some trouble with stuff built with gcc 4.0 and gcc 4.12009-09-02 18:24:48
<osman> i'll try to run this executable under ubuntu (in vmware)2009-09-02 18:25:58
 we'll see what happen :)2009-09-02 18:26:09
 @toffer: installing gcc to ubuntu is same as what i've done (you can see the article)?2009-09-02 18:26:48
<Skymmer> Hi people! And sorry for frequent connect\disconnect. I'm just experimenting with NickServ :)2009-09-02 18:27:07
<osman> something like that i mean "tar -jxvf cygwin-gcc-linux.tar.bz2"2009-09-02 18:27:28
 hi skymmer2009-09-02 18:27:33
<toffer> i guess your build environment is rather archaic2009-09-02 18:27:33
 gcc 2.95 ?!2009-09-02 18:27:36
 that's stoneage2009-09-02 18:27:40
<osman> no no. i have downloaded the other package 3.3.6. i know it's not new either2009-09-02 18:28:19
*** Skymmer has left the channel2009-09-02 18:28:41
 skymmer died again :)2009-09-02 18:28:58
<toffer> as i said you can get trouble with these libraries2009-09-02 18:29:25
 to clarify: you need to have the library which is used on the target platform available for you cross compiler environment2009-09-02 18:31:08
 e.g. the libc, stdc++, etc...2009-09-02 18:31:27
*** Skymmer has joined the channel2009-09-02 18:31:37
 the crt stuff at least2009-09-02 18:31:43
<osman> i've tried in ubuntu. at least it's more informative :)2009-09-02 18:32:07
 osman@olap-ubuntu:~/Desktop$ ./bitarc2009-09-02 18:32:13
 bash: ./bitarc: Permission denied2009-09-02 18:32:19
 osman@olap-ubuntu:~/Desktop$ chmod +x bitarc2009-09-02 18:32:26
 osman@olap-ubuntu:~/Desktop$ ./bitarc2009-09-02 18:32:32
 ./bitarc: error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory2009-09-02 18:32:38
 so, you are right ;)2009-09-02 18:33:08
<toffer> you need to find out which libraries are used there2009-09-02 18:33:47
<osman> so, toffer all of those linux binaries are incompatible between each linux distro? or every programmer make static compile of their own programs?2009-09-02 18:34:07
<Shelwien> yeah, frequently2009-09-02 18:34:51
<toffer> some stuff which has to work everywhere is linked statically (e.g. it was thelike for some nvidia drivers)2009-09-02 18:34:52
 but normally there's more than a single version of a library installed2009-09-02 18:35:07
*** Skymmer has left the channel2009-09-02 18:35:42
*** Skymmer has joined the channel2009-09-02 18:35:59
<osman> so, it's somehow equilavent to ms libraries in a way - usually you cannot any msvc compiled program under the other computers unless you don't statically compiled it or install vcdistr2009-09-02 18:36:14
 *you cannot run2009-09-02 18:36:36
<Shelwien> its easier in MS case actually2009-09-02 18:36:50
 as there're no redistributables for linux usually2009-09-02 18:37:01
<toffer> but keep in mind that there's only a single windows "distro"2009-09-02 18:37:38
 so it's not that hard to have different libs. installed on a system... i bet you find several mscv*.dll versions in you windows folder...2009-09-02 18:38:23
<Skymmer> @toffer: optimizer still running. 155 iters so far2009-09-02 18:38:34
<toffer> nice :)2009-09-02 18:41:24
<Skymmer> I've made a test with best_360.txt and intermediate best_enwik.txt: 23 339 558 for best_360.txt and 23 311 822 for best_enwik.txt2009-09-02 18:41:27
<toffer> i still need to put a release version together an clean the code a bit ... and i can make some graphs comparing 0.3b and 0.42009-09-02 18:42:48
 but still i was busy the whole day2009-09-02 18:43:01
<osman> under ubuntu, i've done this:2009-09-02 18:45:35
 osman@olap-ubuntu:~/Desktop$ ldd bitarc2009-09-02 18:45:36
  linux-gate.so.1 => (0xffffe000)2009-09-02 18:45:43
  libstdc++.so.5 => not found2009-09-02 18:45:49
  libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7f9b000)2009-09-02 18:45:55
  libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7f8f000)2009-09-02 18:46:01
  libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e45000)2009-09-02 18:46:06
  /lib/ld-linux.so.2 (0xb7fcc000)2009-09-02 18:46:11
 but cygwin does not support even ldd command %)2009-09-02 18:46:25
<toffer> well it's just not installed i guess2009-09-02 18:46:49
<osman> anyway. seems libstdc++ didn't exist2009-09-02 18:47:18
<toffer> it's pretty old2009-09-02 18:47:24
 the libstdc++ you linked against2009-09-02 18:47:32
<osman> seems i'll compile all of them under ubuntu2009-09-02 18:47:47
<toffer> well2009-09-02 18:47:54
 you just need to use (build) more recent libraries you links against2009-09-02 18:48:14
 afaik ubuntu is a debian derivative2009-09-02 18:48:26
<osman> yes. it's a debian derivation2009-09-02 18:48:43
<toffer> you may want to try: apt-get install libstdc++52009-09-02 18:48:52
 to get the old c library2009-09-02 18:48:56
 to see that it works2009-09-02 18:49:00
 afterwards get a more recent build environment2009-09-02 18:49:10
<osman> in ubuntu?2009-09-02 18:49:12
 or cygwin?2009-09-02 18:49:19
<Shelwien> i don't think there's apt in cygwin2009-09-02 18:49:58
<toffer> i mean in ubuntu2009-09-02 18:50:10
 he needs to get the library version he used to link against at your cross compile environment2009-09-02 18:50:32
 since the build environment is ancient2009-09-02 18:50:53
 it's no wonder actually2009-09-02 18:50:58
<osman> osman@olap-ubuntu:~/Desktop$ sudo apt-get install libstdc++.so.52009-09-02 18:51:17
 [sudo] password for osman:2009-09-02 18:51:23
<toffer> drop the extensions2009-09-02 18:51:26
<osman> Reading package lists... Done2009-09-02 18:51:29
<toffer> libstdc++52009-09-02 18:51:34
<osman> Building dependency tree 2009-09-02 18:51:37
 ah..ok2009-09-02 18:51:45
 wait2009-09-02 18:51:46
 Building dependency tree2009-09-02 18:52:10
 Reading state information... Done 2009-09-02 18:52:17
 E: Couldn't find package libstdc 2009-09-02 18:52:25
 same as w/o extension2009-09-02 18:52:39
<toffer> ...2009-09-02 18:52:44
 i've already typed in what you need to do2009-09-02 18:52:54
 <toffer> you may want to try: apt-get install libstdc++52009-09-02 18:52:59
 if the packages are named differently in ubuntu2009-09-02 18:53:38
 you can find the name vie apt-cache search libstdc2009-09-02 18:53:48
 via2009-09-02 18:53:56
 got it?2009-09-02 18:54:39
<osman> libstdc++6-4.1-dev - The GNU Standard C++ Library v3 (development files)2009-09-02 18:54:54
 libstdc++6 - The GNU Standard C++ Library v3 2009-09-02 18:55:02
<toffer> cat /etc/apt/sources.list2009-09-02 18:56:10
 so they don't have an old libstdc++ in their package manager?2009-09-02 18:56:47
<osman> very long list have been listed in terminal2009-09-02 18:57:23
<toffer> so is libstdc++5 in there?2009-09-02 18:57:43
 if yes just install it2009-09-02 18:57:47
 but actually you shouldn't modify the target environment to match the build environment. it's like you want to compile a dos app and swap from win to dos just for the build.2009-09-02 18:58:49
<osman> # See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to # newer versions of the distribution. # Line commented out by installer because it failed to verify: #deb http://tr.archive.ubuntu.com/ubuntu/ gutsy main restricted 2009-09-02 18:59:13
 all of the lines commented out2009-09-02 18:59:23
 toffer. seems it's better to use native environment to compile something :)2009-09-02 18:59:50
<toffer> did apt-cache search libstdc++ print out libstdc++52009-09-02 18:59:53
 you an run into the same troubles2009-09-02 19:00:04
 can2009-09-02 19:00:06
<osman> no. as i posted before. there is only libstdc++62009-09-02 19:01:00
*** Simon|B2 has left the channel2009-09-02 21:19:42
*** sami has left the channel2009-09-02 21:59:14
*** toffer has left the channel2009-09-03 00:15:29
<Shelwien> !next2009-09-03 03:47:34