Opened 12 years ago
Last modified 18 months ago
#305 new enhancement
Slow writing to binary streams
Reported by: | stassats | Owned by: | Mark Evenson |
---|---|---|---|
Priority: | major | Milestone: | 1.9.3 |
Component: | streams | Version: | 1.3.0-dev |
Keywords: | Cc: | stassats@… | |
Parent Tickets: |
Description
Slow writing to binary streams
(defun test () (with-open-file (stream "/dev/null" :direction :output :if-exists :overwrite :element-type '(unsigned-byte 8)) (time (loop repeat 10000000 do (write-byte 23 stream)))))
(test)
2.065 seconds real time
30000260 cons cells
The problem boils down to write-byte doing (typep 23 '(unsigned-byte 8)), and typep calls normalize-type, which conses a new type each time: (list 'integer 0 (1- (expt 2 8)))
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/src/org/armedbear/lisp/byte-io.lisp#L34
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/src/org/armedbear/lisp/early-defuns.lisp#L155
Just changing that part to
(UNSIGNED-BYTE (return-from normalize-type (if (or (null i) (eq (car i) '*)) '(integer 0 *) (case (car i) (8 '(integer 0 255)) (16 '(integer 0 65535)) (32 '(integer 0 4294967295)) (64 '(integer 0 18446744073709551615)) (t (list 'integer 0 (1- (expt 2 (car i)))))))))
gives
(test)
1.682 seconds real time
45 cons cells
But even better solution would be to assign efficient writing and reading routines to streams during a stream creation. When I change the body of write-byte to
(defun write-byte (byte pstream) (declare (type stream stream)) (write-8-bits byte stream))
I get
(test)
0.521 seconds real time
12 cons cells
And by extension this problem also affects TYPEP,
(time (typep 23 '(unsigned-byte 8))) => 3 cons cells
(time (typep #(a) '(simple-vector 2))) => 5 cons cells
Change History (19)
comment:1 Changed 11 years ago by
Milestone: | → 1.3.0 |
---|---|
Version: | → 1.3.0-dev |
comment:2 Changed 11 years ago by
comment:3 Changed 11 years ago by
Priority: | minor → major |
---|
comment:6 Changed 11 years ago by
Milestone: | 2.0.0 → 1.4.0 |
---|
comment:11 Changed 5 years ago by
Milestone: | 1.6.2 → 1.7.0 |
---|
comment:16 Changed 3 years ago by
Milestone: | 1.8.1 → 1.9.0 |
---|
comment:17 Changed 22 months ago by
Milestone: | 1.9.0 → 1.9.1 |
---|
comment:18 Changed 22 months ago by
Milestone: | 1.9.1 → 1.9.2 |
---|
comment:19 Changed 18 months ago by
Milestone: | 1.9.2 → 1.9.3 |
---|
Internal ABCL IO needs be redone using the ByteBuffer? objects provided by NIO.
An estimated task of several days to hack, and then a couple weeks to debug.