How does the JVM custom class loader load all classes and jars under the specified classPath?

The content of this article is about how the JVM custom class loader loads all classes and jars under the specified classPath. It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you. help.

1. Class loader types in JVM

From the perspective of the Java virtual machine, there are only two different class loaders: startup class loader and other class loaders.

1. Boot Class Loader (Boostrap ClassLoader): This is implemented by c and is mainly responsible for the core api in the JAVA_HOME/lib directory or the jar package specified by the -Xbootclasspath option.

2. Other class loaders: implemented by java, and their Class objects can be found in the method area. This is further subdivided into several loaders

a). Extension ClassLoader: Responsible for loading the JAVA_HOME/lib/ext directory, or by the -Djava.ext.dirs system variable Specify all class libraries (jars) in the specified path, and developers can directly use the extension class loader. The path specified by the java.ext.dirs system variable can be viewed through System.getProperty("java.ext.dirs").

b). Application ClassLoader: Responsible for loading classes and jar packages in the directory pointed to by java -classpath or -Djava.class.path. Developers can use this class loader directly. This is the program's default loader when no custom class loader is specified.

c). Custom class loader (User ClassLoader): During the running of the program, class files are dynamically loaded through the subclass of java.lang.ClassLoader, reflecting the dynamic real-time class loading characteristics of java.

The hierarchical relationship of these four class loaders is as shown in the figure below.

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　

2. Why do you need to customize the class loader?

Distinguish between classes with the same name: Assume that in tomcat Application servers have many independent applications deployed on them, and they have many classes with the same name but different versions. To distinguish different versions of classes, of course each application needs to have its own independent class loader, otherwise it will be impossible to distinguish which one is used.

1. Class library sharing: Each web application can use its own version of jar in tomcat. But there are things like Servlet-api.jar, java native packages and custom-added Java class libraries that can be shared with each other.

2. Enhance the class: The class loader can rewrite and overwrite the class when loadingClass, during which the functionality of the class can be enhanced. For example, use javassist to add and modify functions to classes, or add dynamic proxies used in aspect-oriented programming, as well as debugging and other principles.

3. Hot replacement: Upgrade the software while the application is running, without restarting the application. For example, JSP update and replacement in toccat server

4. 3. Custom class loader

3.1 Description of related methods of ClassLoader to implement custom class loader

To implement a custom class loader, you need to inherit ClassLoader first. The ClassLoader class is an abstract class responsible for loading objects of classes. You need to know at least three of the methods in custom ClassLoader: loadClass, findClass, defineClass.​

public Class<?> loadClass(String name) throws ClassNotFoundException {return loadClass(name, false);

protected Class<?> findClass(String name) throws ClassNotFoundException {throw new ClassNotFoundException(name);
}

protected final Class<?> defineClass(String name, byte[] b, int off, int len)throws ClassFormatError
{return defineClass(name, b, off, len, null);
}

loadClass

: When the JVM loads a class, it loads the class through the loadClass() method of ClassLoader. loadClass uses the parent delegation mode. If you want to change the parent delegation mode, you can modify loadClass to change the way the class is loaded. The parental delegation model will not be described in detail here.

findClass: ClassLoader loads classes through the findClass() method. The custom class loader implements this method to load the required classes, such as files under the specified path, byte streams, etc.
definedClass: definedClass is used in findClass. By calling the byte array passed in a Class file, a Class object can be generated in the method area, which means findClass implements the class loading function.
Post a section of the loadClass source code in ClassLoader to see its true face...

protected Class<?> loadClass(String name, boolean resolve)
    throws ClassNotFoundException
{
    synchronized (getClassLoadingLock(name)) {
        // First, check if the class has already been loaded
        Class<?> c = findLoadedClass(name);
        if (c == null) {
            long t0 = System.nanoTime();
            try {
                if (parent != null) {
                    c = parent.loadClass(name, false);
                } else {
                    c = findBootstrapClassOrNull(name);
                }
            } catch (ClassNotFoundException e) {
                // ClassNotFoundException thrown if class not found
                // from the non-null parent class loader
            }

            if (c == null) {
                // If still not found, then invoke findClass in order
                // to find the class.
                long t1 = System.nanoTime();
                c = findClass(name);

                // this is the defining class loader; record the stats
                sun.misc.PerfCounter.getParentDelegationTime().addTime(t1 - t0);
                sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(t1);
                sun.misc.PerfCounter.getFindClasses().increment();
            }
        }
        if (resolve) {
            resolveClass(c);
        }
        return c;
    }
}

Source code description...

/**
* Loads the class with the specified <a href="#name">binary name</a>. The
* default implementation of this method searches for classes in the
* following order:
*
* <ol>
*
* <li><p> Invoke {@link #findLoadedClass(String)} to check if the class
* has already been loaded. </p></li>
*
* <li><p> Invoke the {@link #loadClass(String) <tt>loadClass</tt>} method
* on the parent class loader. If the parent is <tt>null</tt> the class
* loader built-in to the virtual machine is used, instead. </p></li>
*
* <li><p> Invoke the {@link #findClass(String)} method to find the
* class. </p></li>
*
* </ol>
*
* <p> If the class was found using the above steps, and the
* <tt>resolve</tt> flag is true, this method will then invoke the {@link
* #resolveClass(Class)} method on the resulting <tt>Class</tt> object.
*
* <p> Subclasses of <tt>ClassLoader</tt> are encouraged to override {@link
* #findClass(String)}, rather than this method. </p>
*
* <p> Unless overridden, this method synchronizes on the result of
* {@link #getClassLoadingLock <tt>getClassLoadingLock</tt>} method
* during the entire class loading process.
*
* @param name
* The <a href="#name">binary name</a> of the class
*
* @param resolve
* If <tt>true</tt> then resolve the class
*
* @return The resulting <tt>Class</tt> object
*
* @throws ClassNotFoundException
* If the class could not be found
*/

The translation is probably

: Use the specified binary name to load the class. The default implementation of this method searches for classes in the following order: Call the findLoadedClass(String) method to check whether this class has been loaded. Use the parent loader to call the loadClass(String) method. If the parent is loaded is Null, the class loader loads the virtual machine's built-in loader and calls the findClass(String) method to load the class. If the corresponding class is successfully found according to the above steps, and the value of the resolve parameter received by this method is true, then Call the resolveClass(Class) method to handle the class. Subclasses of ClassLoader are better off overriding findClass(String) instead of this method. Unless overridden, this method is synchronous (thread-safe) throughout the entire loading process by default.

resolveClass：Class载入必须链接（link），链接指的是把单一的Class加入到有继承关系的类树中。这个方法给Classloader用来链接一个类，如果这个类已经被链接过了，那么这个方法只做一个简单的返回。否则，这个类将被按照 Java™规范中的Execution描述进行链接。

3.2 自定义类加载器实现

按照3.1的说明，继承ClassLoader后重写了findClass方法加载指定路径上的class。先贴上自定义类加载器。

package com.chenerzhu.learning.classloader;

import java.nio.file.Files;
import java.nio.file.Paths;

/**
 * @author chenerzhu
 * @create 2018-10-04 10:47
 **/
public class MyClassLoader extends ClassLoader {
    private String path;

    public MyClassLoader(String path) {
        this.path = path;
    }
    @Override
    protected Class<?> findClass(String name) throws ClassNotFoundException {
        try {
            byte[] result = getClass(name);
            if (result == null) {
                throw new ClassNotFoundException();
            } else {
                return defineClass(name, result, 0, result.length);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

    private byte[] getClass(String name) {
        try {
            return Files.readAllBytes(Paths.get(path));
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

以上就是自定义的类加载器了，实现的功能是加载指定路径的class。再看看如何使用。　

package com.chenerzhu.learning.classloader;
import org.junit.Test;
/**
 * Created by chenerzhu on 2018/10/4.
 */
public class MyClassLoaderTest {
    @Test
    public void testClassLoader() throws Exception {
        MyClassLoader myClassLoader = new MyClassLoader("src/test/resources/bean/Hello.class");
        Class clazz = myClassLoader.loadClass("com.chenerzhu.learning.classloader.bean.Hello");
        Object obj = clazz.newInstance();
        System.out.println(obj);
        System.out.println(obj.getClass().getClassLoader());
    }
}

首先通过构造方法创建MyClassLoader对象myClassLoader，指定加载src/test/resources/bean/Hello.class路径的Hello.class（当然这里只是个例子，直接指定一个class的路径了）。然后通过myClassLoader方法loadClass加载Hello的Class对象，最后实例化对象。以下是输出结果，看得出来实例化成功了，并且类加载器使用的是MyClassLoader。

com.chenerzhu.learning.classloader.bean.Hello@2b2948e2
com.chenerzhu.learning.classloader.MyClassLoader@335eadca

四、类Class卸载

JVM中class和Meta信息存放在PermGen space区域（JDK1.8之后存放在MateSpace中）。如果加载的class文件很多，那么可能导致元数据空间溢出。引起java.lang.OutOfMemory异常。对于有些Class我们可能只需要使用一次，就不再需要了，也可能我们修改了class文件，我们需要重新加载 newclass，那么oldclass就不再需要了。所以需要在JVM中卸载（unload）类Class。
　　JVM中的Class只有满足以下三个条件，才能被GC回收，也就是该Class被卸载（unload）：

1. 该类所有的实例都已经被GC。

2. 该类的java.lang.Class对象没有在任何地方被引用。

3. 加载该类的ClassLoader实例已经被GC。

很容易理解，就是要被卸载的类的ClassLoader实例已经被GC并且本身不存在任何相关的引用就可以被卸载了，也就是JVM清除了类在方法区内的二进制数据。

JVM自带的类加载器所加载的类，在虚拟机的生命周期中，会始终引用这些类加载器，而这些类加载器则会始终引用它们所加载的类的Class对象。因此这些Class对象始终是可触及的，不会被卸载。而用户自定义的类加载器加载的类是可以被卸载的。虽然满足以上三个条件Class可以被卸载，但是GC的时机我们是不可控的，那么同样的我们对于Class的卸载也是不可控的。

五、JVM自定义类加载器加载指定classPath下的所有class及jar

经过以上几个点的说明，现在可以实现JVM自定义类加载器加载指定classPath下的所有class及jar了。这里没有限制class和jar的位置，只要是classPath路径下的都会被加载进JVM，而一些web应用服务器加载是有限定的，比如tomcat加载的是每个应用classPath+“/classes”加载class，classPath+“/lib”加载jar。以下就是代码啦...

package com.chenerzhu.learning.classloader;

import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Enumeration;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;

/**
 * @author chenerzhu
 * @create 2018-10-04 12:24
 **/
public class ClassPathClassLoader extends ClassLoader{

    private static Map<String, byte[]> classMap = new ConcurrentHashMap<>();
    private String classPath;

    public ClassPathClassLoader() {
    }

    public ClassPathClassLoader(String classPath) {
        if (classPath.endsWith(File.separator)) {
            this.classPath = classPath;
        } else {
            this.classPath = classPath + File.separator;
        }
        preReadClassFile();
        preReadJarFile();
    }

    public static boolean addClass(String className, byte[] byteCode) {
        if (!classMap.containsKey(className)) {
            classMap.put(className, byteCode);
            return true;
        }
        return false;
    }

    /**
     * 这里仅仅卸载了myclassLoader的classMap中的class,虚拟机中的
     * Class的卸载是不可控的
     * 自定义类的卸载需要MyClassLoader不存在引用等条件
     * @param className
     * @return
     */
    public static boolean unloadClass(String className) {
        if (classMap.containsKey(className)) {
            classMap.remove(className);
            return true;
        }
        return false;
    }

    /**
     * 遵守双亲委托规则
     */
    @Override
    protected Class<?> findClass(String name) {
        try {
            byte[] result = getClass(name);
            if (result == null) {
                throw new ClassNotFoundException();
            } else {
                return defineClass(name, result, 0, result.length);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

    private byte[] getClass(String className) {
        if (classMap.containsKey(className)) {
            return classMap.get(className);
        } else {
            return null;
        }
    }

    private void preReadClassFile() {
        File[] files = new File(classPath).listFiles();
        if (files != null) {
            for (File file : files) {
                scanClassFile(file);
            }
        }
    }

    private void scanClassFile(File file) {
        if (file.exists()) {
            if (file.isFile() && file.getName().endsWith(".class")) {
                try {
                    byte[] byteCode = Files.readAllBytes(Paths.get(file.getAbsolutePath()));
                    String className = file.getAbsolutePath().replace(classPath, "")
                            .replace(File.separator, ".")
                            .replace(".class", "");
                    addClass(className, byteCode);
                } catch (IOException e) {
                    e.printStackTrace();
                }
            } else if (file.isDirectory()) {
                for (File f : file.listFiles()) {
                    scanClassFile(f);
                }
            }
        }
    }
    private void preReadJarFile() {
        File[] files = new File(classPath).listFiles();
        if (files != null) {
            for (File file : files) {
                scanJarFile(file);
            }
        }
    }

    private void readJAR(JarFile jar) throws IOException {
        Enumeration<JarEntry> en = jar.entries();
        while (en.hasMoreElements()) {
            JarEntry je = en.nextElement();
            je.getName();
            String name = je.getName();
            if (name.endsWith(".class")) {
                //String className = name.replace(File.separator, ".").replace(".class", "");
                String className = name.replace("\\", ".")
                        .replace("/", ".")
                        .replace(".class", "");
                InputStream input = null;
                ByteArrayOutputStream baos = null;
                try {
                    input = jar.getInputStream(je);
                    baos = new ByteArrayOutputStream();
                    int bufferSize = 1024;
                    byte[] buffer = new byte[bufferSize];
                    int bytesNumRead = 0;
                    while ((bytesNumRead = input.read(buffer)) != -1) {
                        baos.write(buffer, 0, bytesNumRead);
                    }
                    addClass(className, baos.toByteArray());
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    if (baos != null) {
                        baos.close();
                    }
                    if (input != null) {
                        input.close();
                    }
                }
            }
        }
    }

    private void scanJarFile(File file) {
        if (file.exists()) {
            if (file.isFile() && file.getName().endsWith(".jar")) {
                try {
                    readJAR(new JarFile(file));
                } catch (IOException e) {
                    e.printStackTrace();
                }
            } else if (file.isDirectory()) {
                for (File f : file.listFiles()) {
                    scanJarFile(f);
                }
            }
        }
    }


    public void addJar(String jarPath) throws IOException {
        File file = new File(jarPath);
        if (file.exists()) {
            JarFile jar = new JarFile(file);
            readJAR(jar);
        }
    }
}

如何使用的代码就不贴了，和3.2节自定义类加载器的使用方式一样。只是构造方法的参数变成classPath了，篇末有代码。当创建MyClassLoader对象时，会自动添加指定classPath下面的所有class和jar里面的class到classMap中，classMap维护className和classCode字节码的关系，只是个缓冲作用，避免每次都从文件中读取。自定义类加载器每次loadClass都会首先在JVM中找是否已经加载className的类，如果不存在就会到classMap中取，如果取不到就是加载错误了。

六、最后

至此，JVM自定义类加载器加载指定classPath下的所有class及jar已经完成了。这篇博文花了两天才写完，在写的过程中有意识地去了解了许多代码的细节，收获也很多。本来最近仅仅是想实现Quartz控制台页面任务添加支持动态class，结果不知不觉跑到类加载器的坑了，在此也趁这个机会总结一遍。当然以上内容并不能保证正确，所以希望大家看到错误能够指出，帮助我更正已有的认知，共同进步。。。

本文的代码已经上传github：https://github.com/chenerzhu/learning/tree/master/classloader

The above is the detailed content of How does the JVM custom class loader load all classes and jars under the specified classPath?. For more information, please follow other related articles on the PHP Chinese website!