Changeset 12607

Timestamp:

04/15/10 14:27:09 (13 years ago)

Author:

Mark Evenson

Message:

URL pathnames working for OPEN for built-in schemas.

Still need to decide with URI escaping issues, as we currently rely on
the URL Stream handlers to do the right thing. And we still need to
retrofit jar pathname's use of a string to represent a URL.

Updates for URL and jar pathname design documents.

Implemented URL-PATHNAME and JAR-PATHNAME as subtypes of PATHNAME.

Adjusted ABCL-TEST-LISP to use functions provided in
"pathname-test.lisp" in "jar-file.lisp". Added one test for url
pathnames.

Constructor in Java added for a Cons by copying references from the
orignal Cons.

Location:

trunk/abcl

Files:

• abcl.asd (modified) (1 diff)
• doc/design/pathnames/jar-pathnames.markdown (modified) (3 diffs)
• doc/design/pathnames/url-pathnames.markdown (modified) (1 diff)
• src/org/armedbear/lisp/BuiltInClass.java (modified) (4 diffs)
• src/org/armedbear/lisp/Cons.java (modified) (1 diff)
• src/org/armedbear/lisp/FileStream.java (modified) (2 diffs)
• src/org/armedbear/lisp/Lisp.java (modified) (1 diff)
• src/org/armedbear/lisp/Load.java (modified) (2 diffs)
• src/org/armedbear/lisp/Pathname.java (modified) (26 diffs)
• src/org/armedbear/lisp/Symbol.java (modified) (2 diffs)
• src/org/armedbear/lisp/Utilities.java (modified) (2 diffs)
• src/org/armedbear/lisp/ZipCache.java (modified) (4 diffs)
• src/org/armedbear/lisp/pathnames.lisp (modified) (4 diffs)
• test/lisp/abcl/jar-file.lisp (modified) (1 diff)

trunk/abcl/abcl.asd

@@ -36 +36 @@
                       (:file "mop-tests" :depends-on ("mop-tests-setup"))
                       (:file "file-system-tests")
-                      (:file "jar-file")
+                      (:file "jar-file" :depend-on ("pathname-test"))
                       (:file "math-tests")
                       (:file "misc-tests")

trunk/abcl/doc/design/pathnames/jar-pathnames.markdown

@@ -4 +4 @@
     Mark Evenson
     Created:  09 JAN 2010
-    Modified: 16 MAR 2010 
-
-Notes towards sketching an implementation of "jar:" references to be
-contained in Common Lisp `PATHNAMEs` within ABCL.  
+    Modified: 25 MAR 2010 
+
+Notes towards an implementation of "jar:" references to be contained
+in Common Lisp `PATHNAME`s within ABCL.
 
 Goals
@@ -52 +52 @@
     resolve works.
 
-7.  Make jar pathnames work as a valid argument for OPEN.
+7.  Make jar pathnames work as a valid argument for OPEN with
+:DIRECTION :INPUT.
 
 8.  Enable the loading of ASDF systems packaged within jar files.
+
+9.  Enable the matching of jar pathnames with PATHNAME-MATCH-P
+
+        (pathname-match-p 
+          "jar:file:/a/b/some.jar!/a/system/def.asd"
+          "jar:file:/**/*.jar!/**/*.asd")      
+        ==> t
 
 Status
 ------
 
-As of svn r12501, all the above goals have been implemented and tested
-*except* for:
-
-7.  Make jar pathnames work as a valid argument for OPEN.
+As of svn r125??, all the above goals have been implemented and
+tested.
 
 
@@ -68 +74 @@
 --------------
 
-Using PATHNAMES
-
-*   A PATHNAME refering to a file within a JAR is known as a JAR
-    PATHNAME.  It can either refer to the entire JAR file or an entry
-    within the JAR file.
-
-*   A JAR PATHNAME always has a DEVICE which is a proper list.  This
-    distinguishes it from other uses of Pathname.  
-
-*   The DEVICE of a JAR PATHNAME will be a list with either one or two
-    elements.  The first element of the JAR PATHNAME can be either a
-    PATHNAME representing a JAR on the filesystem, or a SimpleString
-    representing a URL.
-
-*   a PATHNAME occuring in the list in the DEVICE of a JAR PATHNAME is
-    known as a DEVICE PATHNAME.
-
-*   If the DEVICE is a String it must be a String that successfully
-    references a URL via the java.net.URL(String) constructor
-
-*   Only the first entry in the the DEVICE list may be a String.
-
-*   Otherwise the the DEVICE PATHAME denotes the PATHNAME of the JAR file
-
-*   The DEVICE PATHNAME list of enclosing JARs runs from outermost to
-    innermost.
+A PATHNAME refering to a file within a JAR is known as a JAR PATHNAME.
+It can either refer to the entire JAR file or an entry within the JAR
+file.
+
+A JAR PATHNAME always has a DEVICE which is a proper list.  This
+distinguishes it from other uses of Pathname.
+
+The DEVICE of a JAR PATHNAME will be a list with either one or two
+elements.  The first element of the JAR PATHNAME can be either a
+PATHNAME representing a JAR on the filesystem, or a SimpleString
+representing a URL.
+
+A PATHNAME occuring in the list in the DEVICE of a JAR PATHNAME is
+known as a DEVICE PATHNAME.
+
+If the DEVICE is a String it must be a String that successfully
+references a URL via the java.net.URL(String) constructor
+
+Only the first entry in the the DEVICE list may be a String.
+
+Otherwise the the DEVICE PATHAME denotes the PATHNAME of the JAR file.
+
+The DEVICE PATHNAME list of enclosing JARs runs from outermost to
+innermost.
     
-*   The DIRECTORY component of a JAR PATHNAME should be a list starting
-    with the :ABSOLUTE keyword.  Even though hierarchial entries in
-    jar files are stored in the form "foo/bar/a.lisp" not
-    "/foo/bar/a.lisp", the meaning of DIRECTORY component better
-    represented as an absolute path.
+The DIRECTORY component of a JAR PATHNAME should be a list starting
+with the :ABSOLUTE keyword.  Even though hierarchial entries in jar
+files are stored in the form "foo/bar/a.lisp" not "/foo/bar/a.lisp",
+the meaning of DIRECTORY component better represented as an absolute
+path.
+
+A jar Pathname has type JAR-PATHNAME, derived from PATHNAME.
 
 BNF

trunk/abcl/doc/design/pathnames/url-pathnames.markdown

@@ -111 +111 @@
 conventions of a URL.
 
+A URL Pathname has type URL-PATHNAME, derived from PATHNAME.
 
 Status

trunk/abcl/src/org/armedbear/lisp/BuiltInClass.java

@@ -114 +114 @@
   public static final BuiltInClass PACKAGE              = addClass(Symbol.PACKAGE);
   public static final BuiltInClass PATHNAME             = addClass(Symbol.PATHNAME);
+  public static final BuiltInClass JAR_PATHNAME         = addClass(Symbol.JAR_PATHNAME);
+  public static final BuiltInClass URL_PATHNAME         = addClass(Symbol.URL_PATHNAME);
   public static final BuiltInClass RANDOM_STATE         = addClass(Symbol.RANDOM_STATE);
   public static final BuiltInClass RATIO                = addClass(Symbol.RATIO);
@@ -179 +181 @@
     addClass(Symbol.FILE_STREAM,
              new StructureClass(Symbol.FILE_STREAM, list(SYSTEM_STREAM)));
+  public static final LispClass JAR_STREAM =
+    addClass(Symbol.JAR_STREAM,
+             new StructureClass(Symbol.JAR_STREAM, list(SYSTEM_STREAM)));
+  public static final LispClass URL_STREAM =
+    addClass(Symbol.URL_STREAM,
+             new StructureClass(Symbol.URL_STREAM, list(SYSTEM_STREAM)));
   public static final LispClass CONCATENATED_STREAM =
     addClass(Symbol.CONCATENATED_STREAM,
@@ -231 +239 @@
     FILE_STREAM.setCPL(FILE_STREAM, SYSTEM_STREAM, STREAM,
                        STRUCTURE_OBJECT, CLASS_T);
+    JAR_STREAM.setCPL(JAR_STREAM, SYSTEM_STREAM, STREAM,
+                      STRUCTURE_OBJECT, CLASS_T);
+    URL_STREAM.setCPL(URL_STREAM, SYSTEM_STREAM, STREAM,
+                      STRUCTURE_OBJECT, CLASS_T);
     FLOAT.setDirectSuperclass(REAL);
     FLOAT.setCPL(FLOAT, REAL, NUMBER, CLASS_T);
@@ -261 +273 @@
     PATHNAME.setDirectSuperclass(CLASS_T);
     PATHNAME.setCPL(PATHNAME, CLASS_T);
+    JAR_PATHNAME.setDirectSuperclass(PATHNAME);
+    JAR_PATHNAME.setCPL(JAR_PATHNAME, PATHNAME, CLASS_T);
+    URL_PATHNAME.setDirectSuperclass(PATHNAME);
+    URL_PATHNAME.setCPL(URL_PATHNAME, PATHNAME, CLASS_T);
     RANDOM_STATE.setDirectSuperclass(CLASS_T);
     RANDOM_STATE.setCPL(RANDOM_STATE, CLASS_T);

trunk/abcl/src/org/armedbear/lisp/Cons.java

@@ -59 +59 @@
     this.car = new SimpleString(name);
     this.cdr = value != null ? value : NULL_VALUE;
+    ++count;
+  }
+
+  public Cons(Cons original) 
+  {
+    Cons rest = original;
+    LispObject result = NIL;
+    while (rest.car() != NIL) {
+      result = result.push(rest.car());
+      if (rest.cdr() == NIL) {
+        result = result.push(NIL);
+        break;
+      }
+      rest = (Cons) rest.cdr();
+    }
+    result = result.nreverse();
+    this.car = result.car();
+    this.cdr = result.cdr();
     ++count;
   }

trunk/abcl/src/org/armedbear/lisp/FileStream.java

@@ -287 +287 @@
                 return type_error(first, Symbol.PATHNAME);
             }
-            if (pathname.isJar()) {
-                error(new FileError("Direct stream input/output on entries in JAR files no currently supported.",
-                                    pathname));
-            }
-
             final LispObject namestring = checkString(second);
             LispObject elementType = third;
@@ -301 +296 @@
                 direction != Keyword.IO)
                 error(new LispError("Direction must be :INPUT, :OUTPUT, or :IO."));
-            try {
-                return new FileStream(pathname, namestring.getStringValue(),
-                                      elementType, direction, ifExists,
-                                      externalFormat);
-            }
-            catch (FileNotFoundException e) {
-                return NIL;
-            }
-            catch (IOException e) {
-                return error(new StreamError(null, e));
+
+            if (pathname.isJar())  {
+                if (direction != Keyword.INPUT) {
+                    error(new FileError("Only direction :INPUT is supported for jar files.", pathname));
+                }
+                try { 
+                    return new JarStream(pathname, namestring.getStringValue(),
+                                         elementType, direction, ifExists,
+                                         externalFormat);
+                } catch (IOException e) {
+                    return error(new StreamError(null, e));
+                }
+            } else if (pathname.isURL()) {
+                if (direction != Keyword.INPUT) {
+                    error(new FileError("Only direction :INPUT is supported for URLs.", pathname));
+                }
+                try { 
+                    return new URLStream(pathname, namestring.getStringValue(),
+                                         elementType, direction, ifExists,
+                                         externalFormat);
+                } catch (IOException e) {
+                    return error(new StreamError(null, e));
+                }
+            } else {
+                try {
+                    return new FileStream(pathname, namestring.getStringValue(),
+                                          elementType, direction, ifExists,
+                                          externalFormat);
+                }
+                catch (FileNotFoundException e) {
+                    return NIL;
+                }
+                catch (IOException e) {
+                    return error(new StreamError(null, e));
+                }
             }
         }

trunk/abcl/src/org/armedbear/lisp/Lisp.java

@@ -1742 +1742 @@
     if (arg instanceof FileStream)
       return ((FileStream)arg).getPathname();
+    if (arg instanceof JarStream)
+      return ((JarStream)arg).getPathname();
+    if (arg instanceof URLStream)
+      return ((URLStream)arg).getPathname();
     type_error(arg, list(Symbol.OR, Symbol.PATHNAME,
-                               Symbol.STRING, Symbol.FILE_STREAM));
+                         Symbol.STRING, Symbol.FILE_STREAM,
+                         Symbol.JAR_STREAM, Symbol.URL_STREAM));
     // Not reached.
     return null;

trunk/abcl/src/org/armedbear/lisp/Load.java

@@ -463 +463 @@
                 if (type.equals(COMPILE_FILE_TYPE)
                     || type.equals(COMPILE_FILE_INIT_FASL_TYPE.toString())) {
-                    thread.bindSpecial(Symbol.LOAD_TRUENAME_FASL, truePathname);
+                    Pathname truenameFasl = new Pathname(truePathname);
+                    thread.bindSpecial(Symbol.LOAD_TRUENAME_FASL, truenameFasl);
                 }
                 if (truePathname.type.getStringValue()
@@ -469 +470 @@
                     && truePathname.isJar()) {
                     if (truePathname.device.cdr() != NIL ) {
-                        // set truename to the enclosing JAR
+                        // We set *LOAD-TRUENAME* to the argument that
+                        // a user would pass to LOAD.
+                        Pathname enclosingJar = (Pathname)truePathname.device.cdr().car();
+                        truePathname.device = new Cons(truePathname.device.car(), NIL);
                         truePathname.host = NIL;
-                        truePathname.directory = NIL;
-                        truePathname.name = NIL;
-                        truePathname.type = NIL;
+                        truePathname.directory = enclosingJar.directory;
+                        if (truePathname.directory.car().equals(Keyword.RELATIVE)) {
+                            truePathname.directory.setCar(Keyword.ABSOLUTE);
+                        }
+                        truePathname.name = enclosingJar.name;
+                        truePathname.type = enclosingJar.type;
                         truePathname.invalidateNamestring();
                     } else {

trunk/abcl/src/org/armedbear/lisp/Pathname.java

@@ -40 +40 @@
 import java.io.FileInputStream;
 import java.net.MalformedURLException;
+import java.net.URI;
+import java.net.URISyntaxException;
 import java.net.URL;
 import java.net.URLDecoder;
+import java.net.URLConnection;
 import java.util.Enumeration;
 import java.util.StringTokenizer;
@@ -65 +68 @@
      *  We could do this with setter/getters, but that choose not to in order to avoid the
      *  performance indirection penalty.
+     * 
+     *  Although, given the number of bugs that crop up when this
+     *  protocol is not adhered to, maybe we should consider it.
      */
     public void invalidateNamestring() {
@@ -79 +85 @@
             } else  if (p.host instanceof Symbol) {
                 host = p.host;
+            } else if (p.host instanceof Cons) {
+                host = new Cons((Cons)p.host);
             } else {
                 Debug.assertTrue(false);
@@ -153 +161 @@
 
     public static boolean isSupportedProtocol(String protocol) {
-        return "jar".equals(protocol) || "file".equals(protocol);
+        // There is no programmatic way to know what protocols will
+        // sucessfully construct a URL, so we check for well known ones...
+        if ("jar".equals(protocol) 
+            || "file".equals(protocol))
+            //            || "http".equals(protocol))  XXX remove this as an optimization
+            {
+                return true;
+            }
+        // ... and try the entire constructor with some hopefully
+        // reasonable parameters for everything else.
+        try {
+            new URL(protocol, "example.org", "foo");
+            return true;
+        }  catch (MalformedURLException e) {
+            return false;
+        }
     }
 
     public Pathname(URL url) {
-        String protocol = url.getProtocol();
-        if (!isSupportedProtocol(protocol)) {
-            error(new LispError("Unsupported URL: '" + url.toString() + "'"));
-        }
-
-        if ("jar".equals(protocol)) {
-            init(url.toString());
-            return;
-        } else if ("file".equals(protocol)) {
+        if ("file".equals(url.getProtocol())) {
             String s;
             try {
@@ -189 +204 @@
                 return;
             }
+        } else {
+            init(url.toString());
+            return;
         }
         error(new LispError("Failed to construct Pathname from URL: "
                             + "'" + url.toString() + "'"));
     }
+
+    static final Symbol SCHEME = internKeyword("SCHEME");
+    static final Symbol AUTHORITY = internKeyword("AUTHORITY");
 
     static final private String jarSeparator = "!/";
@@ -231 +252 @@
             }
         }
-
+        
         // A JAR file
         if (s.startsWith("jar:") && s.endsWith(jarSeparator)) {
@@ -303 +324 @@
             type = p.type;
             version = p.version;
+            return;
+        }
+
+        // A URL 
+        if (isValidURL(s)) {
+            URL url = null;
+            try {
+                url = new URL(s);
+            } catch (MalformedURLException e) {
+                Debug.assertTrue(false);
+            }
+            String scheme = url.getProtocol();
+            Debug.assertTrue(scheme != null);
+            String authority = url.getAuthority();
+            Debug.assertTrue(authority != null);
+
+            host = NIL;
+            host = host.push(SCHEME);
+            host = host.push(new SimpleString(scheme));
+            host = host.push(AUTHORITY);
+            host = host.push(new SimpleString(authority));
+            host = host.nreverse();
+
+            device = NIL;
+            
+            // URI encode necessary characters
+            URI uri = null;
+            try { 
+                uri = url.toURI().normalize();
+            } catch (URISyntaxException e) {
+                error(new LispError("Could not URI escape characters in "
+                                    + "'" + url + "'"
+                                    + " because: " + e));
+            }
+
+            String path = uri.getRawPath();
+            if (path == null) {
+                path = "";
+            } 
+            String query = uri.getRawQuery();
+            if (query != null) {
+                path += "?" + query;
+            }
+            String fragment = uri.getRawFragment();
+            if (fragment != null) {
+                path += "#" + fragment;
+            }
+            Pathname p = new Pathname(path != null ? path : ""); 
+
+            directory = p.directory;
+            name = p.name;
+            type = p.type;
+            
             return;
         }
@@ -447 +521 @@
     @Override
     public LispObject typeOf() {
+        if (isURL()) {
+            return Symbol.URL_PATHNAME;
+        } 
+        if (isJar()) {
+            return Symbol.JAR_PATHNAME;
+        }
         return Symbol.PATHNAME;
     }
@@ -452 +532 @@
     @Override
     public LispObject classOf() {
+        if (isURL()) {
+            return BuiltInClass.URL_PATHNAME;
+        } 
+        if (isJar()) {
+            return BuiltInClass.JAR_PATHNAME;
+        }
         return BuiltInClass.PATHNAME;
     }
@@ -460 +546 @@
             return T;
         }
+        if (type == Symbol.JAR_PATHNAME && isJar()) {
+            return T;
+        }
+        if (type == Symbol.URL_PATHNAME && isURL()) {
+            return T;
+        }
         if (type == BuiltInClass.PATHNAME) {
+            return T;
+        }
+        if (type == BuiltInClass.JAR_PATHNAME && isJar()) {
+            return T;
+        }
+        if (type == BuiltInClass.URL_PATHNAME && isURL()) {
             return T;
         }
@@ -487 +585 @@
         // the namestring." 19.2.2.2.3.1
         if (host != NIL) {
-            Debug.assertTrue(host instanceof AbstractString);
-            if (!(this instanceof LogicalPathname)) {
-                sb.append("\\\\"); //UNC file support; if there's a host, it's a UNC path.
-            }
-            sb.append(host.getStringValue());
-            if (this instanceof LogicalPathname) {
-                sb.append(':');
-            } else {
-                sb.append(File.separatorChar);
+            Debug.assertTrue(host instanceof AbstractString 
+                             || host instanceof Cons);
+            if (host instanceof Cons) {
+                LispObject scheme = Symbol.GETF.execute(host, SCHEME, NIL);
+                LispObject authority = Symbol.GETF.execute(host, AUTHORITY, NIL);
+                Debug.assertTrue(scheme != NIL);
+                sb.append(scheme.getStringValue());
+                sb.append(":");
+                if (authority != NIL) {
+                    sb.append("//");
+                    sb.append(authority.getStringValue());
+                }
+            } else {
+                if (!(this instanceof LogicalPathname)) {
+                    sb.append("\\\\"); //UNC file support; if there's a host, it's a UNC path.
+                }
+                sb.append(host.getStringValue());
+                if (this instanceof LogicalPathname) {
+                    sb.append(':');
+                } else {
+                    sb.append(File.separatorChar);
+                }
             }
         }
@@ -583 +694 @@
             }
         }
-        return namestring = sb.toString();
+        namestring = sb.toString();
+        if (isURL()) {
+            namestring = Utilities.uriEncode(namestring);
+        }
+        return namestring;
     }
 
@@ -644 +759 @@
         p.name = name;
         p.type = type;
+        p.invalidateNamestring();
         String path = p.getNamestring();
         StringBuilder result = new StringBuilder();
@@ -746 +862 @@
                 // We have a namestring. Check for pathname components that
                 // can't be read from the namestring.
-                if (host != NIL || version != NIL) {
+                if ((host != NIL && !isURL())
+                    || version != NIL) 
+                {
                     useNamestring = false;
                 } else if (name instanceof AbstractString) {
@@ -829 +947 @@
     }
 
+    public static boolean isValidURL(String s) {
+        try {
+            URL url = new URL(s);
+        } catch (MalformedURLException e) {
+            return false;
+        }
+        return true;
+    }
+
+    public static URL toURL(Pathname p) {
+        URL url = null;
+        if (!(p.host instanceof Cons)) {
+            Debug.assertTrue(false); // XXX
+        }
+        try {
+            url = new URL(p.getNamestring());
+        } catch (MalformedURLException e) {
+            Debug.assertTrue(false); // XXX
+        }
+        return url;
+    }
+
+    URLConnection getURLConnection() {
+        Debug.assertTrue(isURL());
+        URL url = Pathname.toURL(this);
+        URLConnection result = null;
+        try {
+            result = url.openConnection();
+        } catch (IOException e) {
+            error(new FileError("Failed to open URL connection.",
+                                this));
+        }
+        return result;
+    }
+
     public static Pathname parseNamestring(AbstractString namestring) {
         // Check for a logical pathname host.
         String s = namestring.getStringValue();
-        String h = getHostString(s);
-        if (h != null && LOGICAL_PATHNAME_TRANSLATIONS.get(new SimpleString(h)) != null) {
-            // A defined logical pathname host.
-            return new LogicalPathname(h, s.substring(s.indexOf(':') + 1));
+        if (!isValidURL(s)) {
+            String h = getHostString(s);
+            if (h != null && LOGICAL_PATHNAME_TRANSLATIONS.get(new SimpleString(h)) != null) {
+                // A defined logical pathname host.
+                return new LogicalPathname(h, s.substring(s.indexOf(':') + 1));
+            }
         }
         return new Pathname(s);
     }
 
-    public static Pathname parseNamestring(AbstractString namestring,
-      AbstractString host) {
-        // Look for a logical pathname host in the namestring.
+    // XXX was @return Pathname
+    public static LogicalPathname parseNamestring(AbstractString namestring,
+                                                  AbstractString host) 
+    {
         String s = namestring.getStringValue();
+
+        // Look for a logical pathname host in the namestring.        
         String h = getHostString(s);
         if (h != null) {
@@ -1263 +1421 @@
             }
             case 1:
-                return NIL; // ??? huh? -- ME 20100206
+                return NIL; 
             default:
                 return error(new WrongNumberOfArgumentsException(this));
@@ -1327 +1485 @@
                 }
                 return result;
+            }
+
+            if (pathname.isURL()) {
+                return error(new LispError("Unimplemented.")); // XXX
             }
 
@@ -1442 +1604 @@
 
     public boolean isJar() {
-        if (device instanceof Cons) {
-            return true;
-        }
-        return false;
+        return (device instanceof Cons);
+    }
+
+    // ### PATHNAME-URL-P 
+    private static final Primitive PATHNAME_URL_P = new pf_pathname_url_p();
+    private static class pf_pathname_url_p extends Primitive {
+        pf_pathname_url_p() {
+            super("pathname-url-p", PACKAGE_SYS, true, "pathname",
+                  "Predicate for whether PATHNAME references a URL.");
+        }
+        @Override
+        public LispObject execute(LispObject arg) {
+            Pathname p = coerceToPathname(arg);
+            return p.isURL() ? T : NIL;
+        }
+    }
+
+    public boolean isURL() {
+        return (host instanceof Cons);
     }
 
@@ -1607 +1784 @@
             result.directory = mergeDirectories(p.directory, d.directory);
         }
-
-        // A JAR always has absolute directories
-        // if (result.isJar()
-        //     && result.directory instanceof Cons
-        //     && result.directory.car().equals(Keyword.ABSOLUTE)) {
-        //     if (result.directory.cdr().equals(NIL)) {
-        //         result.directory = NIL;
-        //     } else {
-        //         ((Cons)result.directory).car = Keyword.RELATIVE;
-        //     }
-        // }
 
         if (pathname.name != NIL) {
@@ -1728 +1894 @@
                                        pathname));
         }
-        if (!(pathname.device instanceof Cons)) {
+        if (!(pathname.isJar() || pathname.isURL())) {
             pathname
                 = mergePathnames(pathname,
@@ -1750 +1916 @@
                     return error(new FileError(e.getMessage(), pathname));
                 }
+            }
+        } else if (pathname.isURL()) {
+            if (pathname.getInputStream() != null) {
+                return pathname;
             }
         } else
@@ -1886 +2056 @@
                 }
             }
+        } else if (isURL()) {
+            URL url = toURL(this);
+            try { 
+                result = url.openStream();
+            } catch (IOException e) {
+                error(new FileError("Failed to get InputStream from "
+                                    + "'" + Utilities.escapeFormat(getNamestring()) + "'"
+                                    + ": " + e,
+                                    this));
+            }
         } else {
             File file = Utilities.getFile(this);
@@ -1891 +2071 @@
                 result = new FileInputStream(file);
             } catch (IOException e) {
-                Debug.trace("Failed to get InputStream for read from "
-                                + "'" + getNamestring() + "'"
-                                + ": " + e);
+                error(new FileError("Failed to get InputStream from "
+                                    + "'" + getNamestring() + "'"
+                                    + ": " + e, this));
             }
         }
@@ -1903 +2083 @@
      */
     public long getLastModified() {
-        if (!(device instanceof Cons)) {
+        if (!(isJar() || isURL())) {
             File f = Utilities.getFile(this);
             return f.lastModified();
         }
-        // JAR cases
-        // 0.  JAR from URL 
-        // 1.  JAR
-        // 2.  JAR in JAR
-        // 3.  Entry in JAR
-        // 4.  Entry in JAR in JAR
-        String entryPath = asEntryPath();
-        Cons d = (Cons)device;
-        if (d.cdr().equals(NIL)) {
-            if (entryPath.length() == 0) {
-                LispObject o = d.car();
-                if (o instanceof SimpleString) {
-                    // 0. JAR from URL
-                    // URL u = makeJarURL(o.getStringValue());
-                    // XXX unimplemented
-                    Debug.assertTrue(false);
-                    // URLConnection c = null;
-                    // try {
-                    //   c = u.openConnection();
-                    // } catch(IOException e) {
-                    //   Debug.trace("Failed to open Connection for URL "
-                    //               + "'" + u + "'");
-                    //   return 0;
-                    // }
-                    // c.getLastModified();
-                } else  {  
-                    // 1. JAR
-                    return ((Pathname)o).getLastModified();
-                }
-            } else {
-                // 3. Entry in JAR
-                final ZipEntry entry 
-                    = ZipCache.get(device.car()).getEntry(entryPath);
-                if (entry == null) {
-                    return 0;
-                }
-                final long time = entry.getTime();
-                if (time == -1) {
-                    return 0;
-                }
-                return time;
-            }
-        } else {
-            ZipFile outerJar = ZipCache.get(d.car());
-            if (entryPath.length() == 0) {
-                // 4.  JAR in JAR
-                String jarPath = ((Pathname)d.cdr()).asEntryPath();
-                final ZipEntry entry = outerJar.getEntry(jarPath);
-                final long time = entry.getTime();
-                if (time == -1) {
-                    return 0;
-                }
-                return time;
-            } else {
-                // 5. Entry in JAR in JAR
-                String innerJarPath = ((Pathname)d.cdr()).asEntryPath();
-                ZipEntry entry = outerJar.getEntry(entryPath);
-                ZipInputStream innerJarInputStream
-                    = Utilities.getZipInputStream(outerJar, innerJarPath);
-                ZipEntry innerEntry = Utilities.getEntry(innerJarInputStream,
-                                                         entryPath);
-                long time = innerEntry.getTime();
-                if (time == -1) {
-                    return 0;
-                }
-                return time;
-            }
+
+        if (isJar()) {
+            // JAR cases
+            // 0.  JAR from URL 
+            // 1.  JAR
+            // 2.  JAR in JAR
+            // 3.  Entry in JAR
+            // 4.  Entry in JAR in JAR
+            String entryPath = asEntryPath();
+            Cons d = (Cons)device;
+            if (d.cdr().equals(NIL)) {
+                if (entryPath.length() == 0) {
+                    LispObject o = d.car();
+                    if (o instanceof SimpleString) {
+                        // 0. JAR from URL
+                        // URL u = makeJarURL(o.getStringValue());
+                        // XXX unimplemented
+                        Debug.assertTrue(false);
+                        // URLConnection c = null;
+                        // try {
+                        //   c = u.openConnection();
+                        // } catch(IOException e) {
+                        //   Debug.trace("Failed to open Connection for URL "
+                        //               + "'" + u + "'");
+                        //   return 0;
+                        // }
+                        // c.getLastModified();
+                    } else  {  
+                        // 1. JAR
+                        return ((Pathname)o).getLastModified();
+                    }
+                } else {
+                    // 3. Entry in JAR
+                    final ZipEntry entry 
+                        = ZipCache.get(device.car()).getEntry(entryPath);
+                    if (entry == null) {
+                        return 0;
+                    }
+                    final long time = entry.getTime();
+                    if (time == -1) {
+                        return 0;
+                    }
+                    return time;
+                }
+            } else {
+                ZipFile outerJar = ZipCache.get(d.car());
+                if (entryPath.length() == 0) {
+                    // 4.  JAR in JAR
+                    String jarPath = ((Pathname)d.cdr()).asEntryPath();
+                    final ZipEntry entry = outerJar.getEntry(jarPath);
+                    final long time = entry.getTime();
+                    if (time == -1) {
+                        return 0;
+                    }
+                    return time;
+                } else {
+                    // 5. Entry in JAR in JAR
+                    String innerJarPath = ((Pathname)d.cdr()).asEntryPath();
+                    ZipEntry entry = outerJar.getEntry(entryPath);
+                    ZipInputStream innerJarInputStream
+                        = Utilities.getZipInputStream(outerJar, innerJarPath);
+                    ZipEntry innerEntry = Utilities.getEntry(innerJarInputStream,
+                                                             entryPath);
+                    long time = innerEntry.getTime();
+                    if (time == -1) {
+                        return 0;
+                    }
+                    return time;
+                }
+            }
+        }
+        if (isURL()) {
+            return getURLConnection().getLastModified();
         }
         return 0;
@@ -1995 +2181 @@
                                coerceToPathname(Symbol.DEFAULT_PATHNAME_DEFAULTS.symbolValue()),
                                NIL);
+            if (defaultedPathname.isURL() || defaultedPathname.isJar()) {
+                return new FileError("Cannot mkdir with a " 
+                                     + (defaultedPathname.isURL() ? "URL" : "jar")
+                                     + " Pathname.",
+                                     defaultedPathname);
+            }
+                    
             File file = Utilities.getFile(defaultedPathname);
             return file.mkdir() ? T : NIL;
@@ -2089 +2282 @@
         Symbol.DEFAULT_PATHNAME_DEFAULTS.setSymbolValue(coerceToPathname(obj));
     }
+
 }
 

trunk/abcl/src/org/armedbear/lisp/Symbol.java

@@ -3005 +3005 @@
   public static final Symbol SET_SCHAR =
     PACKAGE_SYS.addExternalSymbol("SET-SCHAR");
+  public static final Symbol JAR_STREAM =
+    PACKAGE_SYS.addExternalSymbol("JAR-STREAM");
+  public static final Symbol URL_STREAM =
+    PACKAGE_SYS.addExternalSymbol("URL-STREAM");
+
 
   // Internal symbols in SYSTEM package.
@@ -3061 +3066 @@
   public static final Symbol JAVA_STACK_FRAME =
     PACKAGE_SYS.addInternalSymbol("JAVA-STACK-FRAME");
+  public static final Symbol JAR_PATHNAME =
+    PACKAGE_SYS.addExternalSymbol("JAR-PATHNAME");
+  public static final Symbol URL_PATHNAME =
+    PACKAGE_SYS.addExternalSymbol("URL-PATHNAME");
 
   // CDR6

trunk/abcl/src/org/armedbear/lisp/Utilities.java

@@ -41 +41 @@
 import java.io.IOException;
 import java.io.InputStream;
+import java.net.URI;
+import java.net.URISyntaxException;
 import java.util.jar.JarFile;
 import java.util.zip.ZipEntry;
@@ -254 +256 @@
 
 
-
+    static String uriEncode(String s) {
+        try {
+            URI uri = new URI("?" + s);
+            return uri.getQuery();
+        } catch (URISyntaxException e) {}
+        return null;
+    }
+
+    static String uriDecode(String s) {
+        try {
+            URI uri = new URI(null, null, null, s, null);
+            return uri.toASCIIString().substring(1);
+        } catch (URISyntaxException e) {}
+        return null;  // Error
+    }
+    
+    static String escapeFormat(String s) {
+        return s.replace("~", "~~");
+    }
 }

trunk/abcl/src/org/armedbear/lisp/ZipCache.java

@@ -112 +112 @@
                     return new ZipFile(f);
                 } catch (ZipException e) {
-                    Debug.trace(e); // XXX
-                    return null;
+                    error(new FileError("Failed to construct ZipFile"
+                                        + " because " + e,
+                                        Pathname.makePathname(f)));
                 } catch (IOException e) {
-                    Debug.trace(e); // XXX
-                    return null;
+                    error(new FileError("Failed to contruct ZipFile"
+                                        + " because " + e,
+                                        Pathname.makePathname(f)));
                 }
             } else {
@@ -186 +188 @@
                     entry.file = new ZipFile(f);
                 } catch (ZipException e) {
-                    Debug.trace(e); // XXX
-                    return null;
+                    error(new FileError("Failed to get cached ZipFile"
+                                        + " because " + e,
+                                        Pathname.makePathname(f)));
                 } catch (IOException e) {
-                    Debug.trace(e); // XXX
-                    return null;
+                    error(new FileError("Failed to get cached ZipFile"
+                                        + " because " + e,
+                                        Pathname.makePathname(f)));
                 }
             } else {
@@ -206 +210 @@
             jarURL = new URL("jar:" + url + "!/");
         } catch (MalformedURLException e) {
-            Debug.trace(e);
-            Debug.assertTrue(false); // XXX
-        }
-        URLConnection connection;
+            error(new LispError("Failed to form a jar: URL from "
+                                + "'" + url + "'" 
+                                + " because " + e));
+        }
+        URLConnection connection = null;
         try {
             connection = jarURL.openConnection();
-        } catch (IOException ex) {
-            Debug.trace("Failed to open "
-                        + "'" + jarURL + "'");
-            return null;
+        } catch (IOException e) {
+            error(new LispError("Failed to open "
+                                + "'" + jarURL + "'"
+                                + " with exception " 
+                                + e));
         }
         if (!(connection instanceof JarURLConnection)) {
-            // XXX
-            Debug.trace("Could not get a URLConnection from " + jarURL);
-            return null;
+            error(new LispError("Could not get a URLConnection from " 
+                                + "'" + jarURL + "'"));
         }
         JarURLConnection jarURLConnection = (JarURLConnection) connection;
@@ -227 +232 @@
             result.file = jarURLConnection.getJarFile();
         } catch (IOException e) {
-            Debug.trace(e);
-            Debug.assertTrue(false); // XXX
+            error(new LispError("Failed to fetch URL "
+                                 + "'" + jarURLConnection + "'"
+                                + " because " + e));
         }
         result.lastModified = jarURLConnection.getLastModified();

trunk/abcl/src/org/armedbear/lisp/pathnames.lisp

@@ -135 +135 @@
   (unless (component-match-p (pathname-host pathname) (pathname-host wildcard) nil)
     (return-from pathname-match-p nil))
+  (when (and (pathname-jar-p pathname) 
+             (pathname-jar-p wildcard))
+    (unless 
+        (every (lambda (value) (not (null value)))
+               (mapcar #'pathname-match-p 
+                       (pathname-device pathname)  
+                       (pathname-device wildcard)))
+      (return-from pathname-match-p nil)))
+  (when (or (and (pathname-jar-p pathname)
+                 (not (pathname-jar-p wildcard)))
+            (and (not (pathname-jar-p pathname))
+                 (pathname-jar-p wildcard)))
+    (return-from pathname-match-p nil))
   (let* ((windows-p (featurep :windows))
          (ignore-case (or windows-p (typep pathname 'logical-pathname))))
     (cond ((and windows-p
+                (not (pathname-jar-p pathname))
+                (not (pathname-jar-p wildcard))
                 (not (component-match-p (pathname-device pathname)
                                         (pathname-device wildcard)
@@ -196 +211 @@
          (error "Unsupported wildcard pattern: ~S" to))))
 
+(defun translate-jar-device (source from to &optional case)
+  (declare (ignore case)) ; FIXME
+  (unless to
+    (return-from translate-jar-device nil))
+  (when (not (= (length source) 
+                (length from)
+                (length to)))
+    (error "Unsupported pathname translation for unequal jar ~
+  references: ~S != ~S != ~S" source from to))
+  (mapcar #'translate-pathname source from to))
 
 (defun translate-directory-components-aux (src from to case)
@@ -269 +294 @@
          (device (if (typep 'to 'logical-pathname)
                      :unspecific
-                     (translate-component (pathname-device source)
-                                          (pathname-device from)
-                                          (pathname-device to))))
+                     (if (pathname-jar-p source)
+                         (translate-jar-device (pathname-device source)
+                                               (pathname-device from)
+                                               (pathname-device to))
+                         (translate-component (pathname-device source)
+                                              (pathname-device from)
+                                              (pathname-device to)))))
          (case   (and (typep source 'logical-pathname)
                       (or (featurep :unix) (featurep :windows))
@@ -389 +418 @@
   (cond ((eq host :unspecific)
          (setf host nil))
+        ((consp host)) ;; A URL 
         (host
          (setf host (canonicalize-logical-host host))))

trunk/abcl/test/lisp/abcl/jar-file.lisp

@@ -321 +321 @@
   (:absolute "c" "d") "foo" "lisp")
 
+(deftest jar-file.pathname-match-p.1
+    (pathname-match-p "jar:file:/a/b/some.jar!/a/system/def.asd"
+                      "jar:file:/**/*.jar!/**/*.asd")
+  t)
+
+(deftest jar-file.pathname-match-p.2
+    (pathname-match-p "/a/system/def.asd"
+                      "jar:file:/**/*.jar!/**/*.asd")
+  nil)
+
+(deftest jar-file.pathname-match-p.3
+    (pathname-match-p "jar:file:/a/b/some.jar!/a/system/def.asd"
+                      "/**/*.asd")
+  nil)
+
+(deftest jar-file.translate-pathname.1
+    (namestring
+     (translate-pathname "jar:file:/a/b/c.jar!/d/e/f.lisp" 
+                         "jar:file:/**/*.jar!/**/*.*" 
+                         "/foo/**/*.*"))
+  "/foo/d/e/f.lisp")
+
+;; URL Pathname tests
+(deftest pathname-url.1
+    (let* ((p #p"http://example.org/a/b/foo.lisp")
+           (host (pathname-host p)))
+      (values 
+       (check-physical-pathname p '(:absolute "a" "b") "foo" "lisp")
+       (and (consp host)
+            (equal (getf host :scheme) 
+                   "http")
+            (equal (getf host :authority)
+                   "example.org"))))
+  (t t))
+
       
-      
-             
-
-       
+