Aws-sdk-php: Glob function support

[ahmadpriatama]

Does PHP glob() function supported?

[mtdowling]

I'm not sure what you're asking. Can you elaborate?

[ahmadpriatama]

I want to get list of directories/files which match to some pattern, like we can do with PHP glob function on local filesystem. But instead of searching on local filesystem, i'm trying to get list of directories/files on S3 bucket. So, after following instruction on S3 Stream Wrapper and calling glob('somePattern'), it just return empty array. I wonder whether glob function is also supported like opendir, readdir, etc.?

[ahmadpriatama]

anyway i had implement a function that emulate glob, written below

/**
 * S3StreamHelper class file.
 *
 * @author Ahmad Priatama <[email protected]>
 * @package application.components.helpers
 * @since 2015.04.26
 */
class S3StreamHelper {

    /**
         * Emulate php glob function
     * from http://www.delorie.com/djgpp/doc/libc/libc_426.html
     */
    public function glob($pattern) {
        $return = [];

        $patternFound = preg_match('(\*|\?|\[.+\])', $pattern, $parentPattern, PREG_OFFSET_CAPTURE);
        if ($patternFound) {
            $parent = dirname(substr($pattern, 0, $parentPattern[0][1] + 1));
            $parentLength = strlen($parent);
            $leftover = substr($pattern, $parentPattern[0][1]);
            if (($index = strpos($leftover, '/')) !== FALSE) {
                $searchPattern = substr($pattern, $parentLength + 1, $parentPattern[0][1] - $parentLength + $index - 1);
            } else {
                $searchPattern = substr($pattern, $parentLength + 1);
            }

            $replacement = [
                '/\*/' => '.*',
                '/\?/' => '.'
            ];
            $searchPattern = preg_replace(array_keys($replacement), array_values($replacement), $searchPattern);

            if (is_dir($parent."/") && ($dh = opendir($parent."/"))) {
                while($dir = readdir($dh)) {
                    if (!in_array($dir, ['.', '..'])) {
                        if (preg_match("/^". $searchPattern ."$/", $dir)) {
                            if ($index === FALSE || strlen($leftover) == $index + 1) {
                                $return[] = $parent . "/" . $dir;
                            } else {
                                if (strlen($leftover) > $index + 1) {
                                    $return = array_merge($return, self::glob("{$parent}/{$dir}" . substr($leftover, $index)));
                                }
                            }
                        }
                    }
                }
            }
        } elseif(is_dir($pattern) || is_file($$pattern)) {
            $return[] = $pattern;
        }

        return $return;
    }
}

[mtdowling]

Using glob with the Amazon S3 stream wrapper is not currently possible due to limitations of glob (and I believe the glob stream wrapper): http://php.net/manual/en/function.glob.php

From the docs:

Note: This function will not work on remote files as the file to be examined must be accessible via the server's filesystem.

[angelinetran]

Thanks @ahmadpriatama for the script. It was a life saver!

[mosesliao]

@ahmadpriatama how do I implement your script for glob? Is it something like glob('s3://<bucket_name>/key/to/*.xml')

[manishram]

@ahmadpriatama That was very helpful. Thanks a lot.

Here is an example of how I implemented it, in case someone finds helpful:

class S3StreamHelper {

    /**
         * Emulate php glob function
     * from http://www.delorie.com/djgpp/doc/libc/libc_426.html
     */
    public function glob($pattern) {
        $return = [];

        $patternFound = preg_match('(\*|\?|\[.+\])', $pattern, $parentPattern, PREG_OFFSET_CAPTURE);
        if ($patternFound) {
            $parent = dirname(substr($pattern, 0, $parentPattern[0][1] + 1));
            $parentLength = strlen($parent);
            $leftover = substr($pattern, $parentPattern[0][1]);
            if (($index = strpos($leftover, '/')) !== FALSE) {
                $searchPattern = substr($pattern, $parentLength + 1, $parentPattern[0][1] - $parentLength + $index - 1);
            } else {
                $searchPattern = substr($pattern, $parentLength + 1);
            }

            $replacement = [
                '/\*/' => '.*',
                '/\?/' => '.'
            ];
            $searchPattern = preg_replace(array_keys($replacement), array_values($replacement), $searchPattern);

            if (is_dir($parent."/") && ($dh = opendir($parent."/"))) {
                while($dir = readdir($dh)) {
                    if (!in_array($dir, ['.', '..'])) {
                        if (preg_match("/^". $searchPattern ."$/", $dir)) {
                            if ($index === FALSE || strlen($leftover) == $index + 1) {
                                $return[] = $parent . "/" . $dir;
                            } else {
                                if (strlen($leftover) > $index + 1) {
                                    $return = array_merge($return, self::glob("{$parent}/{$dir}" . substr($leftover, $index)));
                                }
                            }
                        }
                    }
                }
            }
        } elseif(is_dir($pattern) || is_file($$pattern)) {
            $return[] = $pattern;
        }

        return $return;
    }
}


$helperObj = new S3StreamHelper();
foreach($helperObj->glob("s3://my-bucket/*", GLOB_NOSORT) as $image)   
    {  
        echo "Filename: " . $image . "<br />";      
    }