Opened 12 years ago

Last modified 18 months ago

#305 new enhancement

Slow writing to binary streams

Reported by: stassats Owned by: Mark Evenson
Priority: major Milestone: 1.9.3
Component: streams Version: 1.3.0-dev
Keywords: Cc: stassats@…
Parent Tickets:

Description

Slow writing to binary streams

(defun test ()
  (with-open-file (stream "/dev/null" :direction :output
                                      :if-exists :overwrite
                                      :element-type '(unsigned-byte 8))
    (time (loop repeat 10000000 do (write-byte 23 stream)))))

(test)

2.065 seconds real time
30000260 cons cells

The problem boils down to write-byte doing (typep 23 '(unsigned-byte 8)), and typep calls normalize-type, which conses a new type each time: (list 'integer 0 (1- (expt 2 8)))

http://trac.common-lisp.net/armedbear/browser/trunk/abcl/src/org/armedbear/lisp/byte-io.lisp#L34
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/src/org/armedbear/lisp/early-defuns.lisp#L155

Just changing that part to

(UNSIGNED-BYTE
 (return-from normalize-type
   (if (or (null i) (eq (car i) '*))
       '(integer 0 *)
       (case (car i)
         (8 '(integer 0 255))
         (16 '(integer 0 65535))
         (32 '(integer 0 4294967295))
         (64 '(integer 0 18446744073709551615))
         (t
          (list 'integer 0 (1- (expt 2 (car i)))))))))

gives
(test)
1.682 seconds real time
45 cons cells

But even better solution would be to assign efficient writing and reading routines to streams during a stream creation. When I change the body of write-byte to

(defun write-byte (byte pstream)
  (declare (type stream stream))
  (write-8-bits byte stream))

I get
(test)
0.521 seconds real time
12 cons cells

And by extension this problem also affects TYPEP,
(time (typep 23 '(unsigned-byte 8))) => 3 cons cells
(time (typep #(a) '(simple-vector 2))) => 5 cons cells

Change History (19)

comment:1 Changed 11 years ago by Evenson Not Org

Milestone: 1.3.0
Version: 1.3.0-dev

comment:2 Changed 11 years ago by Mark Evenson

Internal ABCL IO needs be redone using the ByteBuffer? objects provided by NIO.

An estimated task of several days to hack, and then a couple weeks to debug.

comment:3 Changed 11 years ago by Mark Evenson

Priority: minormajor

comment:4 Changed 11 years ago by Mark Evenson

Milestone: 1.3.02.0

Ticket retargeted after milestone closed

comment:5 Changed 11 years ago by Mark Evenson

Milestone: 2.02.0.0

Milestone renamed

comment:6 Changed 11 years ago by Mark Evenson

Milestone: 2.0.01.4.0

comment:7 Changed 8 years ago by Mark Evenson

Milestone: 1.4.01.5.0

Ticket retargeted after milestone closed

comment:8 Changed 7 years ago by Mark Evenson

Milestone: 1.5.01.6.0

Ticket retargeted after milestone closed

comment:9 Changed 5 years ago by Mark Evenson

Milestone: 1.6.01.6.1

Ticket retargeted after milestone closed

comment:10 Changed 5 years ago by Mark Evenson

Milestone: 1.6.11.6.2

Ticket retargeted after milestone closed

comment:11 Changed 5 years ago by Mark Evenson

Milestone: 1.6.21.7.0

comment:12 Changed 5 years ago by Mark Evenson

Milestone: 1.7.01.7.1

Ticket retargeted after milestone closed

comment:13 Changed 4 years ago by Mark Evenson

Milestone: 1.7.11.7.2

Ticket retargeted after milestone closed

comment:14 Changed 4 years ago by Mark Evenson

Milestone: 1.7.21.8.0

Milestone renamed

comment:15 Changed 4 years ago by Mark Evenson

Milestone: 1.8.01.8.1

Ticket retargeted after milestone closed

comment:16 Changed 3 years ago by Mark Evenson

Milestone: 1.8.11.9.0

comment:17 Changed 22 months ago by Mark Evenson

Milestone: 1.9.01.9.1

comment:18 Changed 22 months ago by Mark Evenson

Milestone: 1.9.11.9.2

comment:19 Changed 18 months ago by Mark Evenson

Milestone: 1.9.21.9.3
Note: See TracTickets for help on using tickets.