Opened 4 years ago

Last modified 2 months ago

#305 new enhancement

Slow writing to binary streams

Reported by: sboukarev Owned by: mevenson
Priority: major Milestone: 1.6.0
Component: streams Version: 1.3.0-dev
Keywords: Cc: stassats@…
Parent Tickets:

Description

Slow writing to binary streams

(defun test ()
  (with-open-file (stream "/dev/null" :direction :output
                                      :if-exists :overwrite
                                      :element-type '(unsigned-byte 8))
    (time (loop repeat 10000000 do (write-byte 23 stream)))))

(test)

2.065 seconds real time
30000260 cons cells

The problem boils down to write-byte doing (typep 23 '(unsigned-byte 8)), and typep calls normalize-type, which conses a new type each time: (list 'integer 0 (1- (expt 2 8)))

http://trac.common-lisp.net/armedbear/browser/trunk/abcl/src/org/armedbear/lisp/byte-io.lisp#L34
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/src/org/armedbear/lisp/early-defuns.lisp#L155

Just changing that part to

(UNSIGNED-BYTE
 (return-from normalize-type
   (if (or (null i) (eq (car i) '*))
       '(integer 0 *)
       (case (car i)
         (8 '(integer 0 255))
         (16 '(integer 0 65535))
         (32 '(integer 0 4294967295))
         (64 '(integer 0 18446744073709551615))
         (t
          (list 'integer 0 (1- (expt 2 (car i)))))))))

gives
(test)
1.682 seconds real time
45 cons cells

But even better solution would be to assign efficient writing and reading routines to streams during a stream creation. When I change the body of write-byte to

(defun write-byte (byte pstream)
  (declare (type stream stream))
  (write-8-bits byte stream))

I get
(test)
0.521 seconds real time
12 cons cells

And by extension this problem also affects TYPEP,
(time (typep 23 '(unsigned-byte 8))) => 3 cons cells
(time (typep #(a) '(simple-vector 2))) => 5 cons cells

Subtickets (add)

Change History (8)

comment:1 Changed 4 years ago by https://www.google.com/accounts/o8/id?id=AItOawkYnNNEAO_K40Gp0xROhyjOPgjvIskQ48M

  • Milestone set to 1.3.0
  • Version set to 1.3.0-dev

comment:2 Changed 4 years ago by mevenson

Internal ABCL IO needs be redone using the ByteBuffer? objects provided by NIO.

An estimated task of several days to hack, and then a couple weeks to debug.

comment:3 Changed 4 years ago by mevenson

  • Priority changed from minor to major

comment:4 Changed 3 years ago by mevenson

  • Milestone changed from 1.3.0 to 2.0

Ticket retargeted after milestone closed

comment:5 Changed 3 years ago by mevenson

  • Milestone changed from 2.0 to 2.0.0

Milestone renamed

comment:6 Changed 3 years ago by mevenson

  • Milestone changed from 2.0.0 to 1.4.0

comment:7 Changed 11 months ago by mevenson

  • Milestone changed from 1.4.0 to 1.5.0

Ticket retargeted after milestone closed

comment:8 Changed 2 months ago by mevenson

  • Milestone changed from 1.5.0 to 1.6.0

Ticket retargeted after milestone closed

Note: See TracTickets for help on using tickets.