*** Simon|B has left the channel | 2009-08-27 21:58:17 |
* Shelwien just found that 1d1 decoding was broken and didn't write anything at all on output | 2009-08-27 22:11:59 |
<Shelwien> | nobody notices ;) | 2009-08-27 22:12:13 |
| also, qword version of rc is significantly faster than gnu inline asm %) | 2009-08-27 22:14:36 |
| also, no speed difference between 1d and 1d1 ;) | 2009-08-27 22:19:57 |
| i guess the previous difference was caused by broken decoding ;) | 2009-08-27 22:20:19 |
| ... | 2009-08-27 23:03:55 |
| looks like here's a reason for superior performance of hash-based models ;) | 2009-08-27 23:04:28 |
| i tried aligning smaller tables (o1,o2) by 4096 | 2009-08-27 23:05:04 |
| and got a considerable slowdown (31s->36s) | 2009-08-27 23:05:30 |
| wonder why? probably because the same offsets are frequently accessed in all tables | 2009-08-27 23:06:11 |
| and there're only a few cache lines in a set | 2009-08-27 23:06:31 |
| so hashed table access is apparently faster than direct | 2009-08-27 23:07:13 |
| with the current cache design ;) | 2009-08-27 23:07:22 |
*** toffer has left the channel | 2009-08-27 23:22:33 |
| the program with statically allocated 500M (like ccm_sh) seems to have some interesting side effect | 2009-08-27 23:55:10 |
| my XP effectively hangs when ccm_sh crashes ;) | 2009-08-27 23:55:54 |
<osman> | shelwien, i've used 64 alignment and also grouped SSE stages together to increase cache overlapping | 2009-08-27 23:55:57 |
<Shelwien> | i talked about the reverse thing there | 2009-08-27 23:56:22 |
| that aligning the directly indexed tables can cause a noticeable slowdown ;) | 2009-08-27 23:56:54 |
<osman> | not sure about it. because, afair it gained on BIT | 2009-08-27 23:57:24 |
| i wonder why did you such a large number? | 2009-08-27 23:57:41 |
| 4k? | 2009-08-27 23:57:42 |
| *did you use | 2009-08-27 23:57:51 |
<Shelwien> | its a cpu page | 2009-08-27 23:57:52 |
| there's a sense in doing that | 2009-08-27 23:58:03 |
| due do TLB cache etc | 2009-08-27 23:58:16 |
<osman> | afair, cpu page is used at kernel level. i mean for multitasking | 2009-08-27 23:58:33 |
<Shelwien> | cpu page is used on cpu level ;) | 2009-08-27 23:58:48 |
<osman> | OS injects 4k part of the code into CPU and replace the code with different app code and so on | 2009-08-27 23:59:11 |
<Shelwien> | dunno about that | 2009-08-27 23:59:32 |
| but whole x86 virtual addressing is based on 4k pages | 2009-08-27 23:59:54 |
<osman> | at a time, i've attempt to write a multitasking OS which had to fit in 512 bytes %) | 2009-08-28 00:00:09 |
<Shelwien> | also, the cache somewhat works on a page level too | 2009-08-28 00:00:46 |
<osman> | dunno. as i said, i only know that cpu page is used by OS natively | 2009-08-28 00:01:17 |
<Shelwien> | specifically, the cache line sets are common for all pages | 2009-08-28 00:01:19 |
| its only used by OS | 2009-08-28 00:01:32 |
| because its native to cpu | 2009-08-28 00:01:37 |
| but as i said, there're only 2-4 data cache lines | 2009-08-28 00:02:19 |
| for a given line offset in a page | 2009-08-28 00:02:24 |
| so if you would access multiple 4k-aligned offsets at once | 2009-08-28 00:02:55 |
| like [x], [0x1000+x], [0x2000+x] etc | 2009-08-28 00:03:11 |
<osman> | hold on :) there is another fact. every program which runs on xp or higher OS lives at it's own 2GiB address space %) | 2009-08-28 00:03:47 |
<Shelwien> | you only can have max 4 such accesses at once without cache misses | 2009-08-28 00:03:50 |
| i mean, you can only access 5 bytes by _fixed_ offsets in a loop | 2009-08-28 00:04:28 |
| and have a cache miss on each loop iteration | 2009-08-28 00:04:42 |
| and "own address space", aka virtual addressing, is important too, as i said | 2009-08-28 00:05:23 |
| because a non-4k aligned table | 2009-08-28 00:05:44 |
<osman> | did you try to set it to 64 bytes to really see the effect of that? in BIT, I always tried to group different arrays together to increase overlapping and always used 64 bytes alignment on even very small tables | 2009-08-28 00:06:07 |
<Shelwien> | can be allocated as a bunch of non-adjacent physical memory pages | 2009-08-28 00:06:09 |
| as to array interleaving, i understand that, just don't want to mess with it now for convenience | 2009-08-28 00:07:36 |
<osman> | no just. decrease alignment to 64 bytes at compile & test it again. that's all | 2009-08-28 00:09:04 |
| *just decrease | 2009-08-28 00:09:16 |
<Shelwien> | i decreased it to 4 in the end | 2009-08-28 00:09:42 |
| and that helped ;) | 2009-08-28 00:09:46 |
<osman> | %) | 2009-08-28 00:09:50 |
| it's about overlapping i guess | 2009-08-28 00:10:04 |
<Shelwien> | as i said, its about limited cache set siz | 2009-08-28 00:10:35 |
| *size | 2009-08-28 00:10:38 |
| interleaving the tables probably would be even more helpful | 2009-08-28 00:11:12 |
| but i don't want to do it now | 2009-08-28 00:11:22 |
<osman> | union ModelPool { | 2009-08-28 00:12:55 |
| struct { | 2009-08-28 00:13:00 |
| Model order1[256]; | 2009-08-28 00:13:07 |
| Model order2[256*256]; | 2009-08-28 00:13:14 |
| }; | 2009-08-28 00:13:22 |
| Model Pool[256*256*256]; | 2009-08-28 00:13:29 |
| }; | 2009-08-28 00:13:30 |
| ... | 2009-08-28 00:13:33 |
| CACHE_ALIGN ModelPool Pool; | 2009-08-28 00:13:49 |
| ... | 2009-08-28 00:13:56 |
| i've used something like that | 2009-08-28 00:14:04 |
<Shelwien> | that's unrelated then | 2009-08-28 00:14:29 |
| and you perfectly know yourself | 2009-08-28 00:14:42 |
| that in ccm that's different | 2009-08-28 00:14:51 |
| i'm talking about its sparse tables etc | 2009-08-28 00:15:10 |
<osman> | i actually used this in only SSE part | 2009-08-28 00:15:27 |
| i've written as order1, order2 as an example | 2009-08-28 00:15:43 |
| anyway. i have to eat something until sun-rise :) | 2009-08-28 00:19:12 |
*** FuSiyuan has joined the channel | 2009-08-28 00:40:50 |
<Shelwien> | ok, seems i finally managed to implement the rc memory i/o | 2009-08-28 00:50:22 |
| recovering from such crashes is really not funny | 2009-08-28 00:50:40 |
| ... | 2009-08-28 00:53:09 |
| now, effect from rc memory i/o is very little somehow... | 2009-08-28 00:53:30 |
| probably because of ramdrive, but still... | 2009-08-28 00:54:06 |
<osman> | try to cache with copy /b ... ... just before actual compression... | 2009-08-28 01:23:47 |
| i mean copy /b enwik8 nul | 2009-08-28 01:24:09 |
<Shelwien> | ramdrive? ;) | 2009-08-28 01:24:33 |
<osman> | it doesn't matter | 2009-08-28 01:24:41 |
| it's about OS level | 2009-08-28 01:24:46 |
<Shelwien> | well, it doesn't | 2009-08-28 01:25:00 |
| i run multiple tests and can see it | 2009-08-28 01:25:07 |
*** Skymmer has joined the channel | 2009-08-28 02:37:52 |
| skymmer: lots of versions for you today ;) | 2009-08-28 02:48:00 |
| its already 1d5 atm ;) | 2009-08-28 02:48:22 |
<Skymmer> | :) Strange. Header says nothing about it ??? | 2009-08-28 02:49:49 |
<Shelwien> | didn't upload them, still in progress ;) | 2009-08-28 02:50:17 |
<Skymmer> | Nice :) How soon we can lay our hands on it? | 2009-08-28 02:51:46 |
<Shelwien> | uploaded 1d1-1d4 | 2009-08-28 02:53:35 |
| there's new 1d1 - old one wasn't decoding anything ;) | 2009-08-28 02:53:49 |
| (zero output) | 2009-08-28 02:54:00 |
<FuSiyuan> | Hi~ Shelwien. Are you here? | 2009-08-28 02:57:46 |
<Skymmer> | Yes. Realy. | 2009-08-28 02:57:52 |
<Shelwien> | hi, i was always here ;) | 2009-08-28 02:58:12 |
<Skymmer> | zero output | 2009-08-28 02:58:16 |
<Shelwien> | new one is fixed | 2009-08-28 02:58:35 |
| and there're 1d2-1d4 at the similar links | 2009-08-28 02:58:48 |
<FuSiyuan> | I want to know sth. about how to fit the array into L2. | 2009-08-28 02:59:07 |
<Shelwien> | ? | 2009-08-28 02:59:19 |
<FuSiyuan> | I know little about these. I just remembered Osman told me to use small hash tables can speed up the searching. | 2009-08-28 02:59:59 |
<Shelwien> | its not really about fitting into L2 then ;) | 2009-08-28 03:00:35 |
| L2 is like 1M on average, so its hard to fit anything into it ;) | 2009-08-28 03:01:20 |
<FuSiyuan> | Maybe. I only noticed that accessing in big memory blocks may slow down. But don't know the principle | 2009-08-28 03:01:43 |
<Shelwien> | the main feature which you have to know and use | 2009-08-28 03:02:15 |
| is that cpu always reads whole cache lines on any (uncached) memory read | 2009-08-28 03:02:34 |
<FuSiyuan> | What's the size of a cache line? | 2009-08-28 03:03:07 |
<Shelwien> | so packing the sequentially accessed values into a cache line is usually very helpful | 2009-08-28 03:03:30 |
| 64 i guess | 2009-08-28 03:06:37 |
| 32 on older cpus | 2009-08-28 03:06:37 |
<FuSiyuan> | That's why to read int 8 times randomly is more faster than sequentially? | 2009-08-28 03:06:37 |
<Shelwien> | err... random reads would be slower of course | 2009-08-28 03:06:37 |
<FuSiyuan> | Oh. slip of tongue | 2009-08-28 03:07:08 |
| ;) | 2009-08-28 03:07:08 |
| slower | 2009-08-28 03:07:08 |
<Shelwien> | there's also a thing called prefetch | 2009-08-28 03:07:13 |
*** compbooks has left the channel | 2009-08-28 03:07:32 |
| its when you read a value from memory and discard it | 2009-08-28 03:07:41 |
| then do something unrelated | 2009-08-28 03:07:48 |
*** compbooks has joined the channel | 2009-08-28 03:07:51 |
| and then read that value again and use it | 2009-08-28 03:08:07 |
| seems weird | 2009-08-28 03:08:20 |
| but the idea is that its cached on first access | 2009-08-28 03:08:32 |
| without delaying the code as its value is not used there | 2009-08-28 03:08:58 |
<FuSiyuan> | I have not noticed this. | 2009-08-28 03:09:57 |
<Shelwien> | ppmd actively uses this technique | 2009-08-28 03:10:57 |
| i removed the prefetches from it in ppmd_sh | 2009-08-28 03:11:22 |
| to simplify the code | 2009-08-28 03:11:25 |
| and its considerably slower now | 2009-08-28 03:11:34 |
<FuSiyuan> | Another related: read randomly from large ram block would be slow than small blocks, right? | 2009-08-28 03:11:55 |
<Shelwien> | probably... depends on how you use the values | 2009-08-28 03:12:32 |
<FuSiyuan> | Just like i=*(HT+#######); | 2009-08-28 03:13:18 |
| U32 | 2009-08-28 03:13:23 |
<Shelwien> | not how you read them, but how you use them after that | 2009-08-28 03:13:37 |
| but there's another problem which i mentioned here not long ago | 2009-08-28 03:14:29 |
| if you have a table like this (or multiple adjacent tables): | 2009-08-28 03:15:08 |
| byte tab[5][4096]; | 2009-08-28 03:15:21 |
| and try to run a loop like this: | 2009-08-28 03:16:32 |
| for( i=0,s=0; i<N; i++ ) for( j=0,x=idx[i]; j<5; j++ ) s+=tab[j][x]; | 2009-08-28 03:16:33 |
| mysteriously, you'd have cache misses on each memory read there | 2009-08-28 03:17:02 |
*** Skymmer has left the channel | 2009-08-28 03:17:11 |
| that is, all the memory reads would be very slow | 2009-08-28 03:17:16 |
| that's because of the cache design | 2009-08-28 03:18:04 |
| there's a thing called memory page and its size is 4k on x86-32 | 2009-08-28 03:19:02 |
| and there's a very limited number of available cache lines assigned to the same page offset | 2009-08-28 03:19:49 |
| specifically, 4 on "good" cpus, and 2 on "cheap" cpus | 2009-08-28 03:21:04 |
<FuSiyuan> | Are there any materials or books explaining such tricks? | 2009-08-28 03:21:05 |
<Shelwien> | http://agner.org/optimize as i said | 2009-08-28 03:21:17 |
| obviously, also cpu vendor manuals, but these are less usable | 2009-08-28 03:22:11 |
<FuSiyuan> | ;) | 2009-08-28 03:22:27 |
| I'd read the page before asking more. ;) | 2009-08-28 03:23:15 |
<Shelwien> | if you mean agner's site, its not a page, its practically a book ;) | 2009-08-28 03:23:45 |
<FuSiyuan> | Yeah. I see. | 2009-08-28 03:23:57 |
| many pdfs. | 2009-08-28 03:24:11 |
<Shelwien> | if can try the one on the high-level opt maybe | 2009-08-28 03:24:40 |
| this is a funny page too: http://graphics.stanford.edu/~seander/bithacks.html | 2009-08-28 03:25:43 |
*** Skymmer has joined the channel | 2009-08-28 03:36:41 |
| ok, so it did help, i guess ;) | 2009-08-28 03:40:42 |
<Skymmer> | What you mean exactly :) | 2009-08-28 03:41:59 |
<Shelwien> | your results ;) | 2009-08-28 03:42:24 |
<Skymmer> | Maybe I'm paranoic but new versions make my HDD sound differently comparing to previous ones. Probably my imagination. | 2009-08-28 03:43:42 |
<Shelwien> | well, i mainly changed the i/o method there, so who knows ;) | 2009-08-28 03:44:48 |
<Skymmer> | Hmmm... | 2009-08-28 03:45:05 |
| Just read the history.txt and 1e mentioned there :D | 2009-08-28 03:50:54 |
<Shelwien> | well, technically i'm working on 1e | 2009-08-28 03:51:07 |
| but somehow there were a few distinct intermediate stages ;) | 2009-08-28 03:51:35 |
<Skymmer> | By the way what does RC stands for? Relative call? :))) | 2009-08-28 03:52:26 |
<Shelwien> | rangecoder | 2009-08-28 03:52:36 |
<Skymmer> | Interesting what such lame person like me thinks when he hears "rangecoder" name | 2009-08-28 03:54:55 |
| Probably its a coder which works at some range | 2009-08-28 03:55:24 |
<Shelwien> | it works using ranges | 2009-08-28 03:56:02 |
<Skymmer> | And I suppose the problem of rangecoders is to decide which range to work on? | 2009-08-28 03:57:27 |
<Shelwien> | there's no problem ;) | 2009-08-28 03:58:00 |
| but actually its a variation of arithmetic coding | 2009-08-28 03:58:34 |
| and arithmetic coding works by iteratively selecting a partial interval from given interval | 2009-08-28 03:59:30 |
| and "range" is kinda a synonim to "interval" | 2009-08-28 04:00:00 |
| *synonym | 2009-08-28 04:00:19 |
<Skymmer> | Hmm, iteratively selecting a partial interval from given interval means that it doesn't takes into consideration the whole interval??? ehhmm.. sorry :) | 2009-08-28 04:03:55 |
<Shelwien> | in a way | 2009-08-28 04:04:13 |
| but partial interval in containted in the whole interval | 2009-08-28 04:04:25 |
| *is contained | 2009-08-28 04:04:42 |
<Skymmer> | So it assumes that the whole interval will be quite similar to partial interval? | 2009-08-28 04:07:57 |
| Ahh, no matter. I'm just too lame to talk about such things :) | 2009-08-28 04:09:26 |
<Shelwien> | for example, you have an interval [0..2^8192] which correspondins to 1k bytes of data | 2009-08-28 04:09:33 |
| you can iteratively select subintervals from that | 2009-08-28 04:09:58 |
| and eventually end with a single value | 2009-08-28 04:10:06 |
| which would be arithmetic code of some data | 2009-08-28 04:10:29 |
| basically, entropy coding | 2009-08-28 04:10:42 |
| if subintervals are selected by probability | 2009-08-28 04:11:09 |
<Skymmer> | When I first met compression it were lossless audio codecs. Even at that times I was wondered by one simple question | 2009-08-28 04:21:08 |
| How symmetric and assymetric compressors are differ? Ehhmm, I suppose I was not clear enough. | 2009-08-28 04:23:38 |
| For example FLAC -b 2048 -m -l 32 -e -q 0 -r 0,16 -p --lax -A tukey(0,5) takes MUCH more time to encode then Monkey's Audio (MAC) -c4000 | 2009-08-28 04:24:28 |
<Shelwien> | so? | 2009-08-28 04:25:00 |
<Skymmer> | but it extremly fast at decoding, while MAC takes the same time at decoding. | 2009-08-28 04:25:24 |
| let me explain | 2009-08-28 04:25:30 |
| I still don't understand what makes them so different. I suppose that MAC need to do the same work as encoding but in reverse order? But why FLAC don't need so? I mean where is that MAIN factor of assymetrity\symmetrity? Hope you understand :) | 2009-08-28 04:28:24 |
<Shelwien> | there's no main factor | 2009-08-28 04:29:07 |
| the difference is mainly causes by statistical and "ad hoc" approaches | 2009-08-28 04:29:50 |
| *caused | 2009-08-28 04:29:56 |
| like, nobody said that replacing the matching strings with references | 2009-08-28 04:30:55 |
| would allow to compress anything | 2009-08-28 04:31:02 |
| so LZ is not an adaptive algorithm by itself | 2009-08-28 04:32:45 |
| its just a fixed transformation | 2009-08-28 04:33:03 |
| similarly to BWT | 2009-08-28 04:33:10 |
| and its just so happens that many common files can be compressed using that | 2009-08-28 04:33:40 |
| but its a fixed transformation | 2009-08-28 04:33:54 |
| so, such non-adaptive algorithms are commonly asymmetric by nature | 2009-08-28 04:34:33 |
| and asymmetry is multiplied by "external" attempts to add adaptivity by trying multiple versions of such transformations | 2009-08-28 04:35:34 |
| (like LZ optimization or coeff bruteforcing in LPC audio coders) | 2009-08-28 04:36:07 |
| and statistical algorithms are symmetric by nature | 2009-08-28 04:36:47 |
| its just simpler to implement them as symmetric | 2009-08-28 04:37:18 |
| because simple decoder is almost the same as encoder | 2009-08-28 04:37:43 |
| and then nobody cares to further work on that | 2009-08-28 04:37:55 |
| but actually nobody said that statistical compressors (CM etc) can't be asymmetric as well | 2009-08-28 04:38:23 |
| a good example of that might be m1 - a compressor with built-in optimizer | 2009-08-28 04:39:08 |
| it basically allows you to spend a lot of time to optimize the parameters | 2009-08-28 04:39:30 |
| and then encode the file with somewhat better compression | 2009-08-28 04:39:48 |
| so its asymmetric CM compression | 2009-08-28 04:40:27 |
<Skymmer> | So is it possible to store some kind of clues in compressed output so decompressor will perform faster? Or it will hurt ratio considerably? | 2009-08-28 04:41:20 |
<Shelwien> | its exactly what LZMA and flac do | 2009-08-28 04:41:55 |
| and CM compressors can do that too, but their developers are lazy | 2009-08-28 04:42:24 |
<Skymmer> | My gosh! It basicly means that the whole generation of CM developers are not care about the quality of behavior? | 2009-08-28 04:45:25 |
| :) | 2009-08-28 04:45:42 |
<Shelwien> | its more like they have so much work with the main part (the model) | 2009-08-28 04:46:50 |
| that they never reach that stage | 2009-08-28 04:47:22 |
| also there's too much difference between speed stages | 2009-08-28 04:48:02 |
| it doesn't scale that easily | 2009-08-28 04:48:12 |
| and even the fastest statistical algorithms would have complexity similar to that of advanced LZ etc | 2009-08-28 04:48:54 |
| so ideally it would be good to write 10 or more CM codecs based on the versions of the same engine | 2009-08-28 04:49:54 |
| and switch them on encoding or something | 2009-08-28 04:50:07 |
| but that's too much work | 2009-08-28 04:50:19 |
| and people write at most 2 such codecs (see ccm/ccmx and ppmd/ppmonstr) | 2009-08-28 04:50:42 |
| also, don't forget that it only affects the speed | 2009-08-28 04:51:40 |
<Skymmer> | and also nobody cares I guess. I mean target audience. Can be wrong here though. | 2009-08-28 04:52:55 |
<Shelwien> | sure | 2009-08-28 04:53:27 |
| and anyway, that's too much work for just the speed optimization | 2009-08-28 04:53:39 |
| different formats should have the priority | 2009-08-28 04:54:06 |
| and there're so many formats that it never ends ;) | 2009-08-28 04:54:19 |
<Skymmer> | Its good I think. Like, "the dream is valuable until it becomes the reality" | 2009-08-28 04:58:19 |
<Shelwien> | ...IntelC is crazy... | 2009-08-28 05:07:17 |
| there's that thing | 2009-08-28 05:08:09 |
<Skymmer> | Yeah? Why? | 2009-08-28 05:08:26 |
<Shelwien> | x'=((x<<8)+c)*Z | 2009-08-28 05:08:34 |
| y'=((y<<4)+c)*Z | 2009-08-28 05:08:39 |
| so it "optimized" that | 2009-08-28 05:08:58 |
| into | 2009-08-28 05:08:59 |
| z=c*Z | 2009-08-28 05:09:05 |
| x'-(x<<8)*Z+z | 2009-08-28 05:09:15 |
| y'=(y<<4)*Z+z | 2009-08-28 05:09:23 |
| so, added a multiplication %) | 2009-08-28 05:10:01 |
| just to show that it understands commutativity %) | 2009-08-28 05:10:55 |
<Skymmer> | Is it possible to avoid this somehow? Beside patching the code ;) | 2009-08-28 05:12:02 |
<Shelwien> | dunno | 2009-08-28 05:13:45 |
| also i found that it doesn't want to use CMOV instructions | 2009-08-28 05:13:56 |
| with /arch:ia32 | 2009-08-28 05:14:01 |
| only uses it with SSE2 | 2009-08-28 05:14:20 |
| even though that's a damned pentium pro instruction | 2009-08-28 05:15:23 |
| huh?! | 2009-08-28 05:20:13 |
| i rewritten it as | 2009-08-28 05:20:26 |
| (x<<=8)+=c; | 2009-08-28 05:20:35 |
| x*=Z; | 2009-08-28 05:20:40 |
| (same for y) | 2009-08-28 05:20:45 |
| and now there're 2 multiplications %) | 2009-08-28 05:20:54 |
<Skymmer> | Wouldn't you like to report this issue to developers to make a product better? | 2009-08-28 05:21:24 |
<Shelwien> | i have 5 damned compiler errors hanging on their bugtracker | 2009-08-28 05:21:59 |
| for half a year | 2009-08-28 05:22:04 |
| so forget about speed optimizations ;) | 2009-08-28 05:22:20 |
<Skymmer> | Eeehhh, blya :() | 2009-08-28 05:23:06 |
<Shelwien> | yeah, including one with a message "internal error" ;) | 2009-08-28 05:23:31 |
| (there're two actually, but i didn't report the last) | 2009-08-28 05:23:49 |
| btw, do you have SSE2? | 2009-08-28 05:27:13 |
<Skymmer> | Super Superstition Efficiency? | 2009-08-28 05:28:27 |
| :))) | 2009-08-28 05:28:33 |
| Yes. | 2009-08-28 05:28:37 |
| Even SSE3 as reported by CBId | 2009-08-28 05:30:06 |
| and CPU-Z too | 2009-08-28 05:30:56 |
<Shelwien> | but its amd? | 2009-08-28 05:37:09 |
<Skymmer> | yes | 2009-08-28 05:37:28 |
<Shelwien> | so guess i'd have to patch it... | 2009-08-28 05:37:50 |
<Skymmer> | iccpatch ? | 2009-08-28 05:38:08 |
<Shelwien> | kinda; my own version | 2009-08-28 05:38:34 |
<Skymmer> | Really? what is it? | 2009-08-28 05:39:09 |
<Shelwien> | a bat script ;) | 2009-08-28 05:39:40 |
<Skymmer> | Nice. And what additional tool that script calls? | 2009-08-28 05:41:45 |
<Shelwien> | spatch | 2009-08-28 05:42:24 |
<Skymmer> | Interesting to compare the patched outputs from iccpatch and your method | 2009-08-28 05:43:51 |
*** pinc has joined the channel | 2009-08-28 05:44:09 |
| binary I mean | 2009-08-28 05:44:14 |
<Shelwien> | different | 2009-08-28 05:45:56 |
| obviously I made it because iccpatch wasn't enough ;) | 2009-08-28 05:46:12 |
<Skymmer> | Don't tease me :) Some missed additional checks ? | 2009-08-28 05:47:34 |
<Shelwien> | not sure already, it was a few years ago | 2009-08-28 05:47:49 |
| afair one check and GenuineIntel strings | 2009-08-28 05:48:04 |
| http://ctxmodel.net/files/fpaq0pv4b2.rar | 2009-08-28 05:50:26 |
| its in there | 2009-08-28 05:50:29 |
| \bin\iccpatch.bat | 2009-08-28 05:50:49 |
<Skymmer> | Thanx! | 2009-08-28 05:56:36 |
| Well, some time ago I've maded a very simple UPX scrambler via aPatch tool. | 2009-08-28 05:57:47 |
| Its called UPXts - UPX tiny scrambler | 2009-08-28 05:58:14 |
| 4 717 bytes | 2009-08-28 05:58:30 |
| http://skymmer.narod.ru/misc/UPXts_01.rar | 2009-08-28 05:58:38 |
| Lame work surely :) | 2009-08-28 05:58:57 |
| Source inside | 2009-08-28 05:59:19 |
| It just cleans the UPX0\1\2(if exsists) section names and cleans the UPX signature completely | 2009-08-28 06:00:44 |
| not only the UPX! one | 2009-08-28 06:00:56 |
| but 37 or 16 byte sig depending on file | 2009-08-28 06:01:17 |
<Shelwien> | better write a "DanS" block decryptor ;) | 2009-08-28 06:01:20 |
| http://www.ntcore.com/Files/richsign.htm | 2009-08-28 06:03:04 |
| by "decrypting" i mean gathering a lot of .lib files from various VS versions (also IntelC etc) | 2009-08-28 06:04:00 |
| and extracting comp.id tags from there | 2009-08-28 06:04:06 |
| and identifying the libraries use in MS-linked executable | 2009-08-28 06:04:42 |
| *used | 2009-08-28 06:04:47 |
<Skymmer> | Don't read that article yet but just opened one EXE file and looked into it. I see some block at offset 0x80 which is 88 bytes long. | 2009-08-28 06:14:08 |
| 2x8 + 18x4 | 2009-08-28 06:14:32 |
<Shelwien> | yeah, its encrypted a little | 2009-08-28 06:14:35 |
| and contains VS and library versions | 2009-08-28 06:14:44 |
<Skymmer> | I mean two 8 byte blocks with mask = ?? ?? ?? ?? ?? ?? ?? 80 which are at beginning and end | 2009-08-28 06:16:07 |
| and 18 four byte block with mask = ?? ?? ?? 80 | 2009-08-28 06:16:52 |
| *blocks | 2009-08-28 06:17:06 |
| Aha. More repetitive values I see | 2009-08-28 06:21:05 |
*** pinc has left the channel | 2009-08-28 06:22:34 |
*** chornobyl has joined the channel | 2009-08-28 06:39:34 |
<Shelwien> | 1d5 and 1d7 up | 2009-08-28 07:09:51 |
<chornobyl> | mem usage? | 2009-08-28 07:10:15 |
<Shelwien> | the same | 2009-08-28 07:10:20 |
| 1d5 is slow, but there's probability caching (failed? alternative implementation) | 2009-08-28 07:11:10 |
*** pinc has joined the channel | 2009-08-28 07:22:38 |
<Skymmer> | Тут все больше и больше подумываю о написании НФ рассказов. В голове я их пишу давно но на практике пока не добрался. | 2009-08-28 07:49:24 |
<Shelwien> | ну а чего тут "подумывать" - пиши да и все ;) | 2009-08-28 07:50:08 |
<Skymmer> | Идей много, но основных пока три. Даже назавания есть | 2009-08-28 07:50:31 |
<Shelwien> | да, это, как известно, главное ;) | 2009-08-28 07:50:53 |
<Skymmer> | "Время Урожая", "Костер", "Моя любимая брюнетка" | 2009-08-28 07:52:14 |
| Ну ладно, не стыди меня :) | 2009-08-28 07:52:42 |
<Shelwien> | да чего там, сам это проходил просто ;) | 2009-08-28 07:54:12 |
<Skymmer> | Я по чеснарю вроде как открываюсь а ты подкалываешь между строк | 2009-08-28 07:55:26 |
| :) | 2009-08-28 07:55:51 |
<chornobyl> | а о чем будут рассказы | 2009-08-28 07:56:20 |
<Shelwien> | что думаю, то и говорю. я и правда и с этим экспериментировал ;) | 2009-08-28 07:57:00 |
<Skymmer> | Ну как о чем? А о чем все рассказы классиков SF? О людях и их месте в галактике, о роботах, об инопланетянах | 2009-08-28 08:00:14 |
| Ну грубо говоря | 2009-08-28 08:00:36 |
<Shelwien> | вот у меня с людьми главная проблема и оказалась ;) | 2009-08-28 08:01:06 |
<chornobyl> | about cool stuff, like robots, and explosions, and maybe even exploding robots | 2009-08-28 08:01:09 |
<Shelwien> | технических идей много, а люди там вроде и не нужны ;) | 2009-08-28 08:01:33 |
<Skymmer> | С уклоном в легкий депрессионизм :) конечно | 2009-08-28 08:01:52 |
<chornobyl> | гы слава роботам | 2009-08-28 08:01:59 |
<Skymmer> | "Град Обреченный" от Стругацких и "1984" от Оруэла для меня в этом плане знаковы | 2009-08-28 08:03:28 |
<chornobyl> | елки, а я из нф только рассказы в технике молодежи читал | 2009-08-28 08:04:45 |
| и то давно | 2009-08-28 08:04:53 |
<Shelwien> | http://lib.rus.ec и вперед ;) | 2009-08-28 08:05:38 |
<Skymmer> | понятно | 2009-08-28 08:06:00 |
<chornobyl> | дело в том что(щас обидятся) не очень то и охота | 2009-08-28 08:06:32 |
<Shelwien> | ну сейчас такой ассортимент видео продукции, что можно понять, в принципе | 2009-08-28 08:09:26 |
| начинать мб и поздно | 2009-08-28 08:09:33 |
| хотя вообще НФ для словарного запаса и кругозора достаточно полезна | 2009-08-28 08:13:17 |
| а то в школе как-то "в слова" играли, я "синхрофазотрон" загадал ;) | 2009-08-28 08:14:07 |
| а остальные потом очень обижались - они такого слова не знали просто, как выяснилось ;) | 2009-08-28 08:14:40 |
| причем присутствовала одна учительница астрономии ;) | 2009-08-28 08:14:57 |
<chornobyl> | неужели и она не знала | 2009-08-28 08:15:38 |
<Shelwien> | угу. больше всех обижалась | 2009-08-28 08:15:46 |
| причем физмат школа была ;) | 2009-08-28 08:16:19 |
<chornobyl> | ну вы не гуманны, так подрывать ее "непоколебимый" авторитет | 2009-08-28 08:16:33 |
<Shelwien> | да кто ж мог себе такое представить ;) | 2009-08-28 08:16:50 |
<Skymmer> | Я тут спорил с одним человеком насчет книг. Он мне доказывал что это все придуманная туфта. Что жить надо в реальности. | 2009-08-28 08:18:06 |
<Shelwien> | хз, реальность часто гораздо нереальней | 2009-08-28 08:18:39 |
| т.е. в реальности есть такая тенденция что никто на ошибках не учится, и одна и та же унылая фигня повторяется бесконечно | 2009-08-28 08:19:18 |
| так что имхо лучше не привыкать, пока последние мозги не заржавели ;) | 2009-08-28 08:20:22 |
<Skymmer> | Конечно. Я ему доказывал что это просто способ развить свое представление о возможных, теоретически конечно, ситуациях и просто покайфовать :) | 2009-08-28 08:20:40 |
<chornobyl> | боюсь что уже заржевели :'( | 2009-08-28 08:21:32 |
<Skymmer> | И как ты сказал, развить свой язык | 2009-08-28 08:21:36 |
<Shelwien> | еще книжки до сих пор остаются самым надежным средством экономии нервов ;) | 2009-08-28 08:21:42 |
<chornobyl> | и я хочу экономию нервов | 2009-08-28 08:22:19 |
<Shelwien> | я обязательно книгу с собой беру если из дому выхожу надолго | 2009-08-28 08:22:39 |
| в транспорте и очередях очень помогает ;) | 2009-08-28 08:22:58 |
| а главное, переупрямить всяких чиновников получается легко | 2009-08-28 08:23:42 |
| у них книжки нет ;) | 2009-08-28 08:23:47 |
| так что я могу ждать гораздо дольше ;) | 2009-08-28 08:24:00 |
<chornobyl> | гы вот это метод | 2009-08-28 08:24:31 |
| а что часто надо переупрямить? | 2009-08-28 08:24:54 |
<Shelwien> | в детстве выработал для попадения "на компьютеры" | 2009-08-28 08:25:00 |
<chornobyl> | не понял | 2009-08-28 08:25:29 |
<Shelwien> | сидел в универе на подоконнике напротив класса и ждал, пока впустят | 2009-08-28 08:25:30 |
<Skymmer> | Вот прочитал тут недавно "Отец-Основатель" от Саймака. Бля. Вроде все просто но вся фишка в последних двух предложениях. И это чувство мурашек от прочитанного. Просто блин хорошо! | 2009-08-28 08:25:42 |
<chornobyl> | так экономит нервы книга вообще или нф только | 2009-08-28 08:26:41 |
<Shelwien> | наверно вообще, но особо не из чего выбирать имхо | 2009-08-28 08:27:15 |
<Skymmer> | Ну это как контекстно смоделируешь я думваю | 2009-08-28 08:27:38 |
<Shelwien> | т.е. должно быть что-то достаточно легкое и увлекательное | 2009-08-28 08:27:39 |
<chornobyl> | а если классику, я в школе много упустил | 2009-08-28 08:27:56 |
| по литературе | 2009-08-28 08:28:05 |
<Shelwien> | ну вот мне классика определенно не подходит | 2009-08-28 08:28:18 |
| особенно школьная | 2009-08-28 08:28:22 |
| там очень редко что-то читабельное | 2009-08-28 08:28:27 |
<chornobyl> | ну вы ведь еще при союзе учились | 2009-08-28 08:28:54 |
<Shelwien> | мне почему-то кажется, что сейчас хуже | 2009-08-28 08:29:05 |
| особенно у нас | 2009-08-28 08:29:11 |
| при союзе хотя бы было, из чего выбирать | 2009-08-28 08:29:23 |
| а сейчас им надо достойно представить украинскую литературу ;) | 2009-08-28 08:29:43 |
| которой в общем-то нет ;) | 2009-08-28 08:29:49 |
<chornobyl> | не расстравайте меня | 2009-08-28 08:30:08 |
<Skymmer> | Ну почему? Гоголь. | 2009-08-28 08:30:11 |
<Shelwien> | да-да, украина родина слонов | 2009-08-28 08:30:30 |
<chornobyl> | я вот кайдашевую семью с удовольствием читал | 2009-08-28 08:30:52 |
| правда после школы | 2009-08-28 08:31:06 |
<Shelwien> | факт тот, что у нас на книжном рынке современную беллетристику на украинском не продают | 2009-08-28 08:32:33 |
| а местные авторы (которых кстати в харькове дофига просто) дружно печатаются в москве | 2009-08-28 08:33:00 |
| и что характерно, при союзе таки (переводную) фантастику на украинском читал | 2009-08-28 08:33:43 |
| Лема скажем, и "приключения электроника" сначала на украинском попались | 2009-08-28 08:34:05 |
| мне книжек не хватало, так что я харчами не перебирал | 2009-08-28 08:34:26 |
| на польском тоже скажем читал ;) | 2009-08-28 08:35:01 |
| причем хз смогу ли сейчас ;) | 2009-08-28 08:35:13 |
<Skymmer> | Почетно | 2009-08-28 08:35:19 |
<Shelwien> | ну сейчас я вместо этого японский выучил ;) | 2009-08-28 08:35:43 |
| жаль, у китайского фонетика алиенская, так что шансов нет ;) | 2009-08-28 08:36:14 |
<Skymmer> | Меня отец как-то в литературном плане воспитал и я благодарен ему за это. Подсовывал вроде невзначай и затянуло. | 2009-08-28 08:37:27 |
<chornobyl> | мда я быдлею | 2009-08-28 08:38:28 |
<Skymmer> | Он и сам сейчас штудирует книжки и приятно с ним иногда потереть насчет этого | 2009-08-28 08:39:00 |
| А Лем да. Никогда не забуду как за один присест прочел "Мир на Земле" | 2009-08-28 08:40:17 |
<Shelwien> | а как насчет "суммы технологий"? ;) | 2009-08-28 08:40:51 |
<Skymmer> | Боюсь, не читал | 2009-08-28 08:41:14 |
<Shelwien> | ну это не совсем фантастика ;) | 2009-08-28 08:42:00 |
<Skymmer> | Где-то я статейку Лема читал где он про Интернет выражался. Я как-то слегка приуныл в своем отношении к нему. | 2009-08-28 08:45:09 |
| Ну это правда на заре было | 2009-08-28 08:45:57 |
<Shelwien> | писатели тоже люди ;) | 2009-08-28 08:47:01 |
| хотя манией величия страдают чаще обычного ;) | 2009-08-28 08:47:19 |
<Skymmer> | И очень часто пьяньчужки. У того же Кларка это часто проскальзывает. | 2009-08-28 08:50:16 |
| Брррууууу | 2009-08-28 08:50:32 |
| Катнера | 2009-08-28 08:50:39 |
| Я имел в виду | 2009-08-28 08:50:55 |
| Тот же Стивен Кинг об этом писал в каком то предисловии к какой-то "Темной Башне" | 2009-08-28 08:53:28 |
| А сейчас я читаю некого Пашу Энтроповича "Сожми меня поглубже". О бНя, полная порнография. | 2009-08-28 09:01:32 |
<Shelwien> | %) | 2009-08-28 09:02:08 |
<Skymmer> | :) | 2009-08-28 09:02:17 |
*** FuSiyuan has left the channel | 2009-08-28 09:18:48 |
*** Simon|B has joined the channel | 2009-08-28 09:19:06 |
*** toffer has joined the channel | 2009-08-28 09:29:28 |
| А вот мой потрепанный стих из эпохи тотального драгса | 2009-08-28 09:31:30 |
| http://skymmer.narod.ru/misc/Black.txt | 2009-08-28 09:31:35 |
| Чернуха :) | 2009-08-28 09:31:46 |
<toffer> | hi | 2009-08-28 09:41:03 |
<Shelwien> | ... | 2009-08-28 09:41:09 |
<chornobyl> | страшненько | 2009-08-28 09:41:55 |
<toffer> | cheers | 2009-08-28 09:42:49 |
<chornobyl> | hello | 2009-08-28 09:43:02 |
<Shelwien> | Ну ни одной рифмы без мата и "иль", видать не трава мне попалась, а гниль ;) | 2009-08-28 09:43:55 |
<Skymmer> | ну я ж говорю - чернуха. | 2009-08-28 09:47:00 |
<Shelwien> | чернуха хорошая у Ларазчука и Столярова | 2009-08-28 09:48:03 |
| в питере видать трава походящая ;) | 2009-08-28 09:48:11 |
| *подходящая | 2009-08-28 09:48:24 |
| toffer: i already reached 1d9, but still no interleaved rcs ;) | 2009-08-28 09:49:48 |
<Skymmer> | В Питере фен подходящий а трава на югах :) | 2009-08-28 09:50:10 |
<Shelwien> | а, точно, там грибы, я забыл | 2009-08-28 09:50:27 |
<toffer> | 8 releases? | 2009-08-28 09:50:35 |
<Shelwien> | more or less | 2009-08-28 09:50:50 |
<Skymmer> | Ээхх, ладно, обтекаю | 2009-08-28 09:51:12 |
<Shelwien> | " - где бы взять кость 1d2... - монета не подойдет? - а, точно!" ;) | 2009-08-28 09:52:04 |
| toffer: 1d5 had probability caching, but somehow its much slower | 2009-08-28 09:55:26 |
<toffer> | you mean a coding stack? | 2009-08-28 09:55:59 |
<Shelwien> | a probability buffer | 2009-08-28 09:56:13 |
<toffer> | well, yes | 2009-08-28 09:56:17 |
| separately model and code | 2009-08-28 09:56:25 |
<Shelwien> | also i just made a "simplified" rc (w/o 64-bit arithmetics) for interleaving | 2009-08-28 09:57:07 |
| but somehow its slower... | 2009-08-28 09:57:12 |
| ...though, i guess, decoding is faster... | 2009-08-28 09:57:32 |
<toffer> | no 64 bit arithmetic? | 2009-08-28 10:00:00 |
<Shelwien> | yeah, cut low to 24 bits | 2009-08-28 10:00:15 |
<toffer> | where's it used despite the single range adjustment? | 2009-08-28 10:00:16 |
<Shelwien> | the same output size though ;) | 2009-08-28 10:00:31 |
<toffer> | i now normalize only if range < 2^16 | 2009-08-28 10:02:07 |
| output size dropped by 10 bytes for enwik7 | 2009-08-28 10:02:19 |
| maybe that'c cause of bitwise processing | 2009-08-28 10:02:37 |
| that's | 2009-08-28 10:02:41 |
<Shelwien> | in my case output is 1 byte smaller actually ;) | 2009-08-28 10:04:42 |
| because flush is smaller by 1 byte ;) | 2009-08-28 10:04:54 |
*** chornobyl has left the channel | 2009-08-28 11:37:59 |
*** Skymmer has left the channel | 2009-08-28 12:25:19 |
| toffer? | 2009-08-28 12:25:22 |
*** Shelwien has left the channel | 2009-08-28 12:34:03 |
*** Shelwien has joined the channel | 2009-08-28 12:34:08 |
<Simon|B> | what was that? | 2009-08-28 12:34:27 |
<Shelwien> | daily disconnect | 2009-08-28 12:34:40 |
<Simon|B> | Shelwien has disconnected (SVSKilled: GHOST command used by Guest9968193) | 2009-08-28 12:35:10 |
<Shelwien> | well, i disconnected | 2009-08-28 12:35:33 |
| then reconnected and killed the ghost | 2009-08-28 12:35:40 |
<Simon|B> | confusing message | 2009-08-28 12:35:55 |
<Shelwien> | irc doesn't understand that you disconnected | 2009-08-28 12:36:01 |
<Simon|B> | yes | 2009-08-28 12:36:10 |
<Shelwien> | the nick continues to hang around | 2009-08-28 12:36:12 |
| until dropped by ping timeout | 2009-08-28 12:36:28 |
*** toffer has left the channel | 2009-08-28 13:49:22 |
| !next | 2009-08-28 14:01:27 |