26 Commits

Author SHA1 Message Date
Arthur Alves
33b7e5e9d3 Catch NullPointerException to avoid crash due to unsupported async disconnects on Android L (which uses an outdated okhttp version) 2015-12-08 18:59:37 +03:00
Alexey Danilov
0aad13f118 fix dividing by zero while loading empty file (#26) 2015-11-06 13:43:08 +03:00
Alexey Danilov
a270460c8d fix fetching content info (#27) 2015-10-29 16:14:20 +03:00
Alexey Danilov
0264944a20 fix self-pinging (#24) 2015-10-28 10:29:10 +03:00
Alexey Danilov
cf69365920 update last version in README 2015-10-23 18:55:19 +03:00
Alexey Danilov
14ac4a98b1 🐛 fix Content-Range header for partial response according to http spec (http://bit.ly/content-range) 2015-10-23 18:53:57 +03:00
Alexey Danilov
c5cf254b74 fix cleaning cache in 'sample' app 2015-10-23 18:07:01 +03:00
Alexey Danilov
e4ab124d57 fix readme 2015-10-23 16:32:48 +03:00
Alexey Danilov
e7f471983d explain how to use cache limits 2015-10-23 15:32:57 +03:00
Alexey Danilov
d32e88f641 add disk usage limits (total cache size, total files, unlimited) #5 2015-10-23 15:05:18 +03:00
Alexey Danilov
55988e278d use jcenter dependency for sample. fix readme 2015-10-20 15:42:18 +03:00
Alexey Danilov
e06e45c75c remove release notes from readme 2015-10-04 00:37:34 +06:00
Alexey Danilov
1d3230ba57 add seeking video support (#21) and fix streaming while caching (#17) 2015-10-04 00:21:16 +06:00
Alexey Danilov
fe9af27f96 add 'code contributions' section to readme 2015-09-29 11:24:47 +06:00
Alexey Danilov
9ffa983f58 hide 'publish.sh' 2015-09-28 11:39:06 +06:00
Alexey Danilov
6125478d27 fix 🐛 available cache percents callback 2015-09-28 11:31:02 +06:00
Alexey Danilov
e135bf0b42 ping proxy after starting to make sure it works fine 2015-09-25 20:46:02 +03:00
Alexey Danilov
edb12b574f fix readme 2015-09-25 17:08:50 +03:00
Alexey Danilov
63ee20f93c fix offline work 2015-09-25 17:07:46 +03:00
Alexey Danilov
88da0aa5c0 fix markdown 2015-09-21 23:50:06 +03:00
Alexey Danilov
3d50eb64c3 update 'whats new' section in readme 2015-09-21 23:46:17 +03:00
Alexey Danilov
f4b9e5c8f5 fix too long file name for cache. Md5FileNameGenerator tests 2015-09-21 23:38:57 +03:00
Alexey Danilov
15c5388f6c tests for url redirections 2015-09-21 21:24:37 +03:00
Relex
8263814aea unit test for url redirect 2015-09-14 11:29:16 +08:00
Relex
1c7cb32a97 Support redirection 2015-09-12 15:50:56 +08:00
Alexey Danilov
f8f19c5a5c optimize featching source data 2015-09-11 15:37:21 +03:00
44 changed files with 1566 additions and 229 deletions

1
.gitignore vendored
View File

@@ -4,6 +4,7 @@
/.idea
/build
/publish.sh
/local.properties
/gradle.properties
/library/build

View File

@@ -5,14 +5,23 @@
Because there is no sense to download video a lot of times while streaming!
`AndroidVideoCache` allows to add caching support to your `VideoView/MediaPlayer`, [ExoPlayer](https://github.com/danikula/ExoPlayer/commit/6110be8559f003f98020ada8c5e09691b67aaff4) or any another player with help of single line!
## Features
- caching to disk during streaming;
- offline work with cached resources;
- partial loading;
- cache limits (max cache size, max files count);
- multiple clients for same url.
Note `AndroidVideoCache` works only with **direct urls** to media file, it [**doesn't support**](https://github.com/danikula/AndroidVideoCache/issues/19) any streaming technology like DASH, SmoothStreaming, HLS.
## How to use?
Just add link to repository and dependency:
Just add dependency (`AndroidVideoCache` is available in jcenter):
```
repositories {
maven { url 'https://dl.bintray.com/alexeydanilov/maven' }
jcenter()
}
dependencies {
compile 'com.danikula:videocache:2.0.7'
compile 'com.danikula:videocache:2.3.3'
}
```
@@ -34,7 +43,7 @@ private HttpProxyCacheServer getProxy() {
```
To guarantee normal work you should use **single** instance of `HttpProxyCacheServer` for whole app.
For example you can store shared proxy on your `Application`:
For example you can store shared proxy in your `Application`:
```java
public class App extends Application {
@@ -47,24 +56,47 @@ public class App extends Application {
}
private HttpProxyCacheServer newProxy() {
FileNameGenerator nameGenerator = new Md5FileNameGenerator(getExternalCacheDir());
return new HttpProxyCacheServer(nameGenerator);
return new HttpProxyCacheServer(this);
}
}
```
or use [simple factory](http://pastebin.com/38uNkgBT).
or use [simple factory](http://pastebin.com/s2fafSYS).
More preferable way is use some dependency injector like [Dagger](http://square.github.io/dagger/).
By default `HttpProxyCacheServer` uses 512Mb for caching files. You can change this value:
```java
private HttpProxyCacheServer newProxy() {
return new HttpProxyCacheServer.Builder(this)
.maxCacheSize(1024 * 1024 * 1024) // 1 Gb for cache
.build();
}
```
or can limit total count of files in cache:
```java
private HttpProxyCacheServer newProxy() {
return new HttpProxyCacheServer.Builder(this)
.maxCacheFilesCount(20)
.build();
}
```
See `sample` app for details.
## Whats new in 2.0?
- simpler api
- single cache for multiple clients
- cache file name policy
- more powerful listener
- more samples
- less log flood
## Whats new
See Release Notes [here](https://github.com/danikula/AndroidVideoCache/releases)
## Code contributions
If it's a feature that you think would need to be discussed please open an issue first, otherwise, you can follow this process:
1. [Fork the project](http://help.github.com/fork-a-repo/)
2. Create a feature branch (git checkout -b my_branch)
3. Push your changes to your new branch (git push origin my_branch)
4. Initiate a [pull request](http://help.github.com/send-pull-requests/) on github
5. Your pull request will be reviewed and hopefully merged :)
## Where published?
[Here](https://bintray.com/alexeydanilov/maven/videocache/view)
@@ -86,4 +118,4 @@ See `sample` app for details.
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
limitations under the License.

View File

@@ -9,6 +9,6 @@ buildscript {
allprojects {
repositories {
mavenCentral()
jcenter()
}
}

View File

@@ -3,4 +3,4 @@ distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-2.4-all.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-2.8-all.zip

View File

@@ -26,7 +26,7 @@ publish {
userOrg = 'alexeydanilov'
groupId = 'com.danikula'
artifactId = 'videocache'
publishVersion = '2.0.7'
publishVersion = '2.3.3'
description = 'Cache support for android VideoView'
website = 'https://github.com/danikula/AndroidVideoCache'
}

View File

@@ -22,7 +22,7 @@ public class ByteArraySource implements Source {
}
@Override
public int available() throws ProxyCacheException {
public int length() throws ProxyCacheException {
return data.length;
}

View File

@@ -0,0 +1,30 @@
package com.danikula.videocache;
import com.danikula.videocache.file.DiskUsage;
import com.danikula.videocache.file.FileNameGenerator;
import java.io.File;
/**
* Configuration for proxy cache.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
class Config {
public final File cacheRoot;
public final FileNameGenerator fileNameGenerator;
public final DiskUsage diskUsage;
Config(File cacheRoot, FileNameGenerator fileNameGenerator, DiskUsage diskUsage) {
this.cacheRoot = cacheRoot;
this.fileNameGenerator = fileNameGenerator;
this.diskUsage = diskUsage;
}
File generateCacheFile(String url) {
String name = fileNameGenerator.generate(url);
return new File(cacheRoot, name);
}
}

View File

@@ -63,9 +63,9 @@ class GetRequest {
@Override
public String toString() {
return "GetRequest{" +
"uri='" + uri + '\'' +
", rangeOffset=" + rangeOffset +
"rangeOffset=" + rangeOffset +
", partial=" + partial +
", uri='" + uri + '\'' +
'}';
}
}

View File

@@ -2,11 +2,15 @@ package com.danikula.videocache;
import android.text.TextUtils;
import com.danikula.videocache.file.FileCache;
import java.io.BufferedOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.net.Socket;
import static com.danikula.videocache.ProxyCacheUtils.DEFAULT_BUFFER_SIZE;
/**
* {@link ProxyCache} that read http url and writes data to {@link Socket}
*
@@ -14,6 +18,8 @@ import java.net.Socket;
*/
class HttpProxyCache extends ProxyCache {
private static final float NO_CACHE_BARRIER = .2f;
private final HttpUrlSource source;
private final FileCache cache;
private CacheListener listener;
@@ -30,30 +36,29 @@ class HttpProxyCache extends ProxyCache {
public void processRequest(GetRequest request, Socket socket) throws IOException, ProxyCacheException {
OutputStream out = new BufferedOutputStream(socket.getOutputStream());
byte[] buffer = new byte[ProxyCacheUtils.DEFAULT_BUFFER_SIZE];
int readBytes;
boolean headersWrote = false;
String responseHeaders = newResponseHeaders(request);
out.write(responseHeaders.getBytes("UTF-8"));
long offset = request.rangeOffset;
while ((readBytes = read(buffer, offset, buffer.length)) != -1) {
// tiny optimization: to prevent HEAD request in source for content-length. content-length 'll available after reading source
if (!headersWrote) {
String responseHeaders = newResponseHeaders(request);
out.write(responseHeaders.getBytes("UTF-8"));
headersWrote = true;
}
out.write(buffer, 0, readBytes);
offset += readBytes;
if (cache.isCompleted()) {
onCacheAvailable(100);
}
if (isUseCache(request)) {
responseWithCache(out, offset);
} else {
responseWithoutCache(out, offset);
}
out.flush();
}
private boolean isUseCache(GetRequest request) throws ProxyCacheException {
int sourceLength = source.length();
boolean sourceLengthKnown = sourceLength > 0;
int cacheAvailable = cache.available();
// do not use cache for partial requests which too far from available cache. It seems user seek video.
return !sourceLengthKnown || !request.partial || request.rangeOffset <= cacheAvailable + sourceLength * NO_CACHE_BARRIER;
}
private String newResponseHeaders(GetRequest request) throws IOException, ProxyCacheException {
String mime = source.getMime();
boolean mimeKnown = !TextUtils.isEmpty(mime);
int length = cache.isCompleted() ? cache.available() : source.available();
int length = cache.isCompleted() ? cache.available() : source.length();
boolean lengthKnown = length >= 0;
long contentLength = request.partial ? length - request.rangeOffset : length;
boolean addRange = lengthKnown && request.partial;
@@ -61,14 +66,40 @@ class HttpProxyCache extends ProxyCache {
.append(request.partial ? "HTTP/1.1 206 PARTIAL CONTENT\n" : "HTTP/1.1 200 OK\n")
.append("Accept-Ranges: bytes\n")
.append(lengthKnown ? String.format("Content-Length: %d\n", contentLength) : "")
.append(addRange ? String.format("Content-Range: bytes %d-%d/%d\n", request.rangeOffset, length, length) : "")
.append(addRange ? String.format("Content-Range: bytes %d-%d/%d\n", request.rangeOffset, length - 1, length) : "")
.append(mimeKnown ? String.format("Content-Type: %s\n", mime) : "")
.append("\n") // headers end
.toString();
}
private void responseWithCache(OutputStream out, long offset) throws ProxyCacheException, IOException {
byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
int readBytes;
while ((readBytes = read(buffer, offset, buffer.length)) != -1) {
out.write(buffer, 0, readBytes);
offset += readBytes;
}
out.flush();
}
private void responseWithoutCache(OutputStream out, long offset) throws ProxyCacheException, IOException {
try {
HttpUrlSource source = new HttpUrlSource(this.source);
source.open((int) offset);
byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
int readBytes;
while ((readBytes = source.read(buffer)) != -1) {
out.write(buffer, 0, readBytes);
offset += readBytes;
}
out.flush();
} finally {
source.close();
}
}
@Override
protected void onCacheAvailable(int percents) {
protected void onCachePercentsAvailableChanged(int percents) {
if (listener != null) {
listener.onCacheAvailable(cache.file, source.url, percents);
}

View File

@@ -1,21 +1,37 @@
package com.danikula.videocache;
import android.content.Context;
import android.os.SystemClock;
import android.util.Log;
import com.danikula.videocache.file.DiskUsage;
import com.danikula.videocache.file.FileNameGenerator;
import com.danikula.videocache.file.Md5FileNameGenerator;
import com.danikula.videocache.file.TotalCountLruDiskUsage;
import com.danikula.videocache.file.TotalSizeLruDiskUsage;
import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketException;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.Callable;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeoutException;
import static com.danikula.videocache.Preconditions.checkAllNotNull;
import static com.danikula.videocache.Preconditions.checkNotNull;
import static com.danikula.videocache.ProxyCacheUtils.LOG_TAG;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
/**
* Simple lightweight proxy server with file caching support that handles HTTP requests.
@@ -39,6 +55,8 @@ import static com.danikula.videocache.ProxyCacheUtils.LOG_TAG;
public class HttpProxyCacheServer {
private static final String PROXY_HOST = "127.0.0.1";
private static final String PING_REQUEST = "ping";
private static final String PING_RESPONSE = "ping ok";
private final Object clientsLock = new Object();
private final ExecutorService socketProcessor = Executors.newFixedThreadPool(8);
@@ -46,10 +64,15 @@ public class HttpProxyCacheServer {
private final ServerSocket serverSocket;
private final int port;
private final Thread waitConnectionThread;
private final FileNameGenerator fileNameGenerator;
private final Config config;
private boolean pinged;
public HttpProxyCacheServer(FileNameGenerator fileNameGenerator) {
this.fileNameGenerator = checkNotNull(fileNameGenerator);
public HttpProxyCacheServer(Context context) {
this(new Builder(context).buildConfig());
}
private HttpProxyCacheServer(Config config) {
this.config = checkNotNull(config);
try {
InetAddress inetAddress = InetAddress.getByName(PROXY_HOST);
this.serverSocket = new ServerSocket(0, 8, inetAddress);
@@ -58,13 +81,64 @@ public class HttpProxyCacheServer {
this.waitConnectionThread = new Thread(new WaitRequestsRunnable(startSignal));
this.waitConnectionThread.start();
startSignal.await(); // freeze thread, wait for server starts
Log.i(LOG_TAG, "Proxy cache server started. Ping it...");
makeSureServerWorks();
} catch (IOException | InterruptedException e) {
socketProcessor.shutdown();
throw new IllegalStateException("Error starting local proxy server", e);
}
}
private void makeSureServerWorks() {
int maxPingAttempts = 3;
int delay = 300;
int pingAttempts = 0;
while (pingAttempts < maxPingAttempts) {
try {
Future<Boolean> pingFuture = socketProcessor.submit(new PingCallable());
this.pinged = pingFuture.get(delay, MILLISECONDS);
if (this.pinged) {
return;
}
SystemClock.sleep(delay);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
Log.e(LOG_TAG, "Error pinging server [attempt: " + pingAttempts + ", timeout: " + delay + "]. ", e);
}
pingAttempts++;
delay *= 2;
}
Log.e(LOG_TAG, "Shutdown server… Error pinging server [attempts: " + pingAttempts + ", max timeout: " + delay / 2 + "]. " +
"If you see this message, please, email me danikula@gmail.com");
shutdown();
}
private boolean pingServer() throws ProxyCacheException {
String pingUrl = appendToProxyUrl(PING_REQUEST);
HttpUrlSource source = new HttpUrlSource(pingUrl);
try {
byte[] expectedResponse = PING_RESPONSE.getBytes();
source.open(0);
byte[] response = new byte[expectedResponse.length];
source.read(response);
boolean pingOk = Arrays.equals(expectedResponse, response);
Log.d(LOG_TAG, "Ping response: `" + new String(response) + "`, pinged? " + pingOk);
return pingOk;
} catch (ProxyCacheException e) {
Log.e(LOG_TAG, "Error reading ping response", e);
return false;
} finally {
source.close();
}
}
public String getProxyUrl(String url) {
if (!pinged) {
Log.e(LOG_TAG, "Proxy server isn't pinged. Caching doesn't work. If you see this message, please, email me danikula@gmail.com");
}
return pinged ? appendToProxyUrl(url) : url;
}
private String appendToProxyUrl(String url) {
return String.format("http://%s:%d/%s", PROXY_HOST, port, ProxyCacheUtils.encode(url));
}
@@ -140,30 +214,51 @@ public class HttpProxyCacheServer {
GetRequest request = GetRequest.read(socket.getInputStream());
Log.i(LOG_TAG, "Request to cache proxy:" + request);
String url = ProxyCacheUtils.decode(request.uri);
HttpProxyCacheServerClients clients = getClients(url);
clients.processRequest(request, socket);
if (PING_REQUEST.equals(url)) {
responseToPing(socket);
} else {
HttpProxyCacheServerClients clients = getClients(url);
clients.processRequest(request, socket);
}
} catch (SocketException e) {
// There is no way to determine that client closed connection http://stackoverflow.com/a/10241044/999458
// So just to prevent log flooding don't log stacktrace
Log.d(LOG_TAG, "Client communication problem. It seems client closed connection");
Log.d(LOG_TAG, "Closing socket… Socket is closed by client.");
} catch (ProxyCacheException | IOException e) {
onError(new ProxyCacheException("Error processing request", e));
} finally {
releaseSocket(socket);
Log.d(LOG_TAG, "Opened connections: " + getClientsCount());
}
}
private void responseToPing(Socket socket) throws IOException {
OutputStream out = socket.getOutputStream();
out.write("HTTP/1.1 200 OK\n\n".getBytes());
out.write(PING_RESPONSE.getBytes());
}
private HttpProxyCacheServerClients getClients(String url) throws ProxyCacheException {
synchronized (clientsLock) {
HttpProxyCacheServerClients clients = clientsMap.get(url);
if (clients == null) {
clients = new HttpProxyCacheServerClients(url, fileNameGenerator);
clients = new HttpProxyCacheServerClients(url, config);
clientsMap.put(url, clients);
}
return clients;
}
}
private int getClientsCount() {
synchronized (clientsLock) {
int count = 0;
for (HttpProxyCacheServerClients clients : clientsMap.values()) {
count += clients.getClientsCount();
}
return count;
}
}
private void releaseSocket(Socket socket) {
closeSocketInput(socket);
closeSocketOutput(socket);
@@ -178,7 +273,7 @@ public class HttpProxyCacheServer {
} catch (SocketException e) {
// There is no way to determine that client closed connection http://stackoverflow.com/a/10241044/999458
// So just to prevent log flooding don't log stacktrace
Log.d(LOG_TAG, "Error closing client's input stream: it seems client closed connection");
Log.d(LOG_TAG, "Releasing input stream… Socket is closed by client.");
} catch (IOException e) {
onError(new ProxyCacheException("Error closing socket input stream", e));
}
@@ -237,4 +332,101 @@ public class HttpProxyCacheServer {
}
}
private class PingCallable implements Callable<Boolean> {
@Override
public Boolean call() throws Exception {
return pingServer();
}
}
/**
* Builder for {@link HttpProxyCacheServer}.
*/
public static final class Builder {
private static final long DEFAULT_MAX_SIZE = 512 * 104 * 1024;
private File cacheRoot;
private FileNameGenerator fileNameGenerator;
private DiskUsage diskUsage;
public Builder(Context context) {
this.cacheRoot = StorageUtils.getIndividualCacheDirectory(context);
this.diskUsage = new TotalSizeLruDiskUsage(DEFAULT_MAX_SIZE);
this.fileNameGenerator = new Md5FileNameGenerator();
}
/**
* Overrides default cache folder to be used for caching files.
* <p/>
* By default AndroidVideoCache uses
* '/Android/data/[app_package_name]/cache/video-cache/' if card is mounted and app has appropriate permission
* or 'video-cache' subdirectory in default application's cache directory otherwise.
* <p/>
* <b>Note</b> directory must be used <b>only</b> for AndroidVideoCache files.
*
* @param file a cache directory, can't be null.
* @return a builder.
*/
public Builder cacheDirectory(File file) {
this.cacheRoot = checkNotNull(file);
return this;
}
/**
* Overrides default cache file name generator {@link Md5FileNameGenerator} .
*
* @param fileNameGenerator a new file name generator.
* @return a builder.
*/
public Builder fileNameGenerator(FileNameGenerator fileNameGenerator) {
this.fileNameGenerator = checkNotNull(fileNameGenerator);
return this;
}
/**
* Sets max cache size in bytes.
* All files that exceeds limit will be deleted using LRU strategy.
* Default value is 512 Mb.
* <p/>
* Note this method overrides result of calling {@link #maxCacheFilesCount(int)}
*
* @param maxSize max cache size in bytes.
* @return a builder.
*/
public Builder maxCacheSize(long maxSize) {
this.diskUsage = new TotalSizeLruDiskUsage(maxSize);
return this;
}
/**
* Sets max cache files count.
* All files that exceeds limit will be deleted using LRU strategy.
* <p/>
* Note this method overrides result of calling {@link #maxCacheSize(long)}
*
* @param count max cache files count.
* @return a builder.
*/
public Builder maxCacheFilesCount(int count) {
this.diskUsage = new TotalCountLruDiskUsage(count);
return this;
}
/**
* Builds new instance of {@link HttpProxyCacheServer}.
*
* @return proxy cache. Only single instance should be used across whole app.
*/
public HttpProxyCacheServer build() {
Config config = buildConfig();
return new HttpProxyCacheServer(config);
}
private Config buildConfig() {
return new Config(cacheRoot, fileNameGenerator, diskUsage);
}
}
}

View File

@@ -4,6 +4,8 @@ import android.os.Handler;
import android.os.Looper;
import android.os.Message;
import com.danikula.videocache.file.FileCache;
import java.io.File;
import java.io.IOException;
import java.net.Socket;
@@ -24,26 +26,33 @@ final class HttpProxyCacheServerClients {
private final String url;
private volatile HttpProxyCache proxyCache;
private final List<CacheListener> listeners = new CopyOnWriteArrayList<>();
private final FileNameGenerator fileNameGenerator;
private final CacheListener uiCacheListener;
private final Config config;
public HttpProxyCacheServerClients(String url, FileNameGenerator fileNameGenerator) {
public HttpProxyCacheServerClients(String url, Config config) {
this.url = checkNotNull(url);
this.fileNameGenerator = checkNotNull(fileNameGenerator);
this.config = checkNotNull(config);
this.uiCacheListener = new UiListenerHandler(url, listeners);
}
public void processRequest(GetRequest request, Socket socket) throws ProxyCacheException, IOException {
proxyCache = proxyCache == null ? newHttpProxyCache() : proxyCache;
startProcessRequest();
try {
clientsCount.incrementAndGet();
proxyCache.processRequest(request, socket);
} finally {
int count = clientsCount.decrementAndGet();
if (count <= 0) {
proxyCache.shutdown();
proxyCache = null;
}
finishProcessRequest();
}
}
private synchronized void startProcessRequest() throws ProxyCacheException {
proxyCache = proxyCache == null ? newHttpProxyCache() : proxyCache;
}
private synchronized void finishProcessRequest() {
if (clientsCount.decrementAndGet() <= 0) {
proxyCache.shutdown();
proxyCache = null;
}
}
@@ -65,9 +74,13 @@ final class HttpProxyCacheServerClients {
clientsCount.set(0);
}
public int getClientsCount() {
return clientsCount.get();
}
private HttpProxyCache newHttpProxyCache() throws ProxyCacheException {
HttpUrlSource source = new HttpUrlSource(url);
FileCache cache = new FileCache(fileNameGenerator.generate(url));
FileCache cache = new FileCache(config.generateCacheFile(url), config.diskUsage);
HttpProxyCache httpProxyCache = new HttpProxyCache(source, cache);
httpProxyCache.registerCacheListener(uiCacheListener);
return httpProxyCache;

View File

@@ -3,15 +3,20 @@ package com.danikula.videocache;
import android.text.TextUtils;
import android.util.Log;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InterruptedIOException;
import java.net.HttpURLConnection;
import java.net.URL;
import static com.danikula.videocache.ProxyCacheUtils.DEFAULT_BUFFER_SIZE;
import static com.danikula.videocache.ProxyCacheUtils.LOG_TAG;
import static java.net.HttpURLConnection.HTTP_MOVED_PERM;
import static java.net.HttpURLConnection.HTTP_MOVED_TEMP;
import static java.net.HttpURLConnection.HTTP_OK;
import static java.net.HttpURLConnection.HTTP_PARTIAL;
import static java.net.HttpURLConnection.HTTP_SEE_OTHER;
/**
* {@link Source} that uses http resource as source for {@link ProxyCache}.
@@ -20,10 +25,11 @@ import static java.net.HttpURLConnection.HTTP_PARTIAL;
*/
public class HttpUrlSource implements Source {
private static final int MAX_REDIRECTS = 5;
public final String url;
private HttpURLConnection connection;
private InputStream inputStream;
private volatile int available = Integer.MIN_VALUE;
private volatile int length = Integer.MIN_VALUE;
private volatile String mime;
public HttpUrlSource(String url) {
@@ -35,42 +41,48 @@ public class HttpUrlSource implements Source {
this.mime = mime;
}
public HttpUrlSource(HttpUrlSource source) {
this.url = source.url;
this.mime = source.mime;
this.length = source.length;
}
@Override
public int available() throws ProxyCacheException {
if (available == Integer.MIN_VALUE) {
public synchronized int length() throws ProxyCacheException {
if (length == Integer.MIN_VALUE) {
fetchContentInfo();
}
return available;
return length;
}
@Override
public void open(int offset) throws ProxyCacheException {
try {
Log.d(LOG_TAG, "Open connection " + (offset > 0 ? " with offset " + offset : "") + " to " + url);
connection = (HttpURLConnection) new URL(url).openConnection();
if (offset > 0) {
connection.setRequestProperty("Range", "bytes=" + offset + "-");
}
connection = openConnection(offset, -1);
mime = connection.getContentType();
inputStream = connection.getInputStream();
available = readSourceAvailableBytes(connection, offset);
inputStream = new BufferedInputStream(connection.getInputStream(), DEFAULT_BUFFER_SIZE);
length = readSourceAvailableBytes(connection, offset, connection.getResponseCode());
} catch (IOException e) {
throw new ProxyCacheException("Error opening connection for " + url + " with offset " + offset, e);
}
}
private int readSourceAvailableBytes(HttpURLConnection connection, int offset) throws IOException {
private int readSourceAvailableBytes(HttpURLConnection connection, int offset, int responseCode) throws IOException {
int contentLength = connection.getContentLength();
int responseCode = connection.getResponseCode();
return responseCode == HTTP_OK ? contentLength :
responseCode == HTTP_PARTIAL ? contentLength + offset :
available;
return responseCode == HTTP_OK ? contentLength
: responseCode == HTTP_PARTIAL ? contentLength + offset : length;
}
@Override
public void close() throws ProxyCacheException {
if (connection != null) {
connection.disconnect();
try {
connection.disconnect();
} catch (NullPointerException e) {
// https://github.com/danikula/AndroidVideoCache/issues/32
// https://github.com/danikula/AndroidVideoCache/issues/29
throw new ProxyCacheException("Error disconnecting HttpUrlConnection", e);
}
}
}
@@ -91,34 +103,65 @@ public class HttpUrlSource implements Source {
private void fetchContentInfo() throws ProxyCacheException {
Log.d(LOG_TAG, "Read content info from " + url);
HttpURLConnection urlConnection = null;
InputStream inputStream = null;
try {
urlConnection = (HttpURLConnection) new URL(url).openConnection();
urlConnection.setRequestMethod("HEAD");
available = urlConnection.getContentLength();
urlConnection = openConnection(0, 10000);
length = urlConnection.getContentLength();
mime = urlConnection.getContentType();
Log.i(LOG_TAG, "Info read: " + this);
inputStream = urlConnection.getInputStream();
Log.i(LOG_TAG, "Content info for `" + url + "`: mime: " + mime + ", content-length: " + length);
} catch (IOException e) {
throw new ProxyCacheException("Error fetching Content-Length from " + url);
Log.e(LOG_TAG, "Error fetching info from " + url, e);
} finally {
ProxyCacheUtils.close(inputStream);
if (urlConnection != null) {
urlConnection.disconnect();
}
}
}
public String getMime() throws ProxyCacheException {
private HttpURLConnection openConnection(int offset, int timeout) throws IOException, ProxyCacheException {
HttpURLConnection connection;
boolean redirected;
int redirectCount = 0;
String url = this.url;
do {
Log.d(LOG_TAG, "Open connection " + (offset > 0 ? " with offset " + offset : "") + " to " + url);
connection = (HttpURLConnection) new URL(url).openConnection();
if (offset > 0) {
connection.setRequestProperty("Range", "bytes=" + offset + "-");
}
if (timeout > 0) {
connection.setConnectTimeout(timeout);
connection.setReadTimeout(timeout);
}
int code = connection.getResponseCode();
redirected = code == HTTP_MOVED_PERM || code == HTTP_MOVED_TEMP || code == HTTP_SEE_OTHER;
if (redirected) {
url = connection.getHeaderField("Location");
redirectCount++;
connection.disconnect();
}
if (redirectCount > MAX_REDIRECTS) {
throw new ProxyCacheException("Too many redirects: " + redirectCount);
}
} while (redirected);
return connection;
}
public synchronized String getMime() throws ProxyCacheException {
if (TextUtils.isEmpty(mime)) {
fetchContentInfo();
}
return mime;
}
public String getUrl() {
return url;
}
@Override
public String toString() {
return "HttpUrlSource{" +
"url='" + url + '\'' +
", available=" + available +
", mime='" + mime + '\'' +
'}';
return "HttpUrlSource{url='" + url + "}";
}
}

View File

@@ -1,56 +0,0 @@
package com.danikula.videocache;
import android.text.TextUtils;
import java.io.File;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import static com.danikula.videocache.Preconditions.checkNotNull;
/**
* Implementation of {@link FileNameGenerator} that uses MD5 of url as file name
*
* @author Alexey Danilov (danikula@gmail.com).
*/
public class Md5FileNameGenerator implements FileNameGenerator {
private final File cacheDirectory;
public Md5FileNameGenerator(File cacheDirectory) {
this.cacheDirectory = checkNotNull(cacheDirectory);
}
@Override
public File generate(String url) {
checkNotNull(url);
String extension = getExtension(url);
String name = computeMD5(url);
name = TextUtils.isEmpty(extension) ? name : name + "." + extension;
return new File(cacheDirectory, name);
}
private String getExtension(String url) {
int dotIndex = url.lastIndexOf('.');
int slashIndex = url.lastIndexOf(File.separator);
return dotIndex != -1 && dotIndex > slashIndex ? url.substring(dotIndex + 1, url.length()) : "";
}
private String computeMD5(String string) {
try {
MessageDigest messageDigest = MessageDigest.getInstance("MD5");
byte[] digestBytes = messageDigest.digest(string.getBytes());
return bytesToHexString(digestBytes);
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException(e);
}
}
private String bytesToHexString(byte[] bytes) {
StringBuffer sb = new StringBuffer();
for (byte b : bytes) {
sb.append(String.format("%02x", b));
}
return sb.toString();
}
}

View File

@@ -24,9 +24,10 @@ class ProxyCache {
private final Cache cache;
private final Object wc = new Object();
private final Object stopLock = new Object();
private final AtomicInteger readSourceErrorsCount;
private volatile Thread sourceReaderThread;
private volatile boolean stopped;
private final AtomicInteger readSourceErrorsCount;
private volatile int percentsAvailable = -1;
public ProxyCache(Source source, Cache cache) {
this.source = checkNotNull(source);
@@ -40,17 +41,14 @@ class ProxyCache {
while (!cache.isCompleted() && cache.available() < (offset + length) && !stopped) {
readSourceAsync();
waitForSourceData();
checkIsCacheValid();
checkReadSourceErrorsCount();
}
return cache.read(buffer, offset, length);
}
private void checkIsCacheValid() throws ProxyCacheException {
int sourceAvailable = source.available();
if (sourceAvailable > 0 && cache.available() > sourceAvailable) {
throw new ProxyCacheException("Unexpected cache: cache [" + cache.available() + " bytes] > source[" + sourceAvailable + " bytes]");
int read = cache.read(buffer, offset, length);
if (cache.isCompleted() && percentsAvailable != 100) {
percentsAvailable = 100;
onCachePercentsAvailableChanged(100);
}
return read;
}
private void checkReadSourceErrorsCount() throws ProxyCacheException {
@@ -76,10 +74,10 @@ class ProxyCache {
}
}
private void readSourceAsync() throws ProxyCacheException {
private synchronized void readSourceAsync() throws ProxyCacheException {
boolean readingInProgress = sourceReaderThread != null && sourceReaderThread.getState() != Thread.State.TERMINATED;
if (!stopped && !cache.isCompleted() && !readingInProgress) {
sourceReaderThread = new Thread(new SourceReaderRunnable(), "Source reader for ProxyCache");
sourceReaderThread = new Thread(new SourceReaderRunnable(), "Source reader for " + source);
sourceReaderThread.start();
}
}
@@ -94,22 +92,35 @@ class ProxyCache {
}
}
private void notifyNewCacheDataAvailable(int cachePercentage) {
onCacheAvailable(cachePercentage);
private void notifyNewCacheDataAvailable(long cacheAvailable, long sourceAvailable) {
onCacheAvailable(cacheAvailable, sourceAvailable);
synchronized (wc) {
wc.notifyAll();
}
}
protected void onCacheAvailable(int percents){
protected void onCacheAvailable(long cacheAvailable, long sourceLength) {
boolean zeroLengthSource = sourceLength == 0;
int percents = zeroLengthSource ? 100 : (int) (cacheAvailable * 100 / sourceLength);
boolean percentsChanged = percents != percentsAvailable;
boolean sourceLengthKnown = sourceLength >= 0;
if (sourceLengthKnown && percentsChanged) {
onCachePercentsAvailableChanged(percents);
}
percentsAvailable = percents;
}
protected void onCachePercentsAvailableChanged(int percentsAvailable) {
}
private void readSource() {
int cachePercentage = 0;
int sourceAvailable = -1;
int offset = 0;
try {
int offset = cache.available();
offset = cache.available();
source.open(offset);
sourceAvailable = source.length();
byte[] buffer = new byte[ProxyCacheUtils.DEFAULT_BUFFER_SIZE];
int readBytes;
while ((readBytes = source.read(buffer)) != -1) {
@@ -120,9 +131,7 @@ class ProxyCache {
cache.append(buffer, readBytes);
}
offset += readBytes;
cachePercentage = offset * 100 / source.available();
notifyNewCacheDataAvailable(cachePercentage);
notifyNewCacheDataAvailable(offset, sourceAvailable);
}
tryComplete();
} catch (Throwable e) {
@@ -130,13 +139,13 @@ class ProxyCache {
onError(e);
} finally {
closeSource();
notifyNewCacheDataAvailable(cachePercentage);
notifyNewCacheDataAvailable(offset, sourceAvailable);
}
}
private void tryComplete() throws ProxyCacheException {
synchronized (stopLock) {
if (!isStopped() && cache.available() == source.available()) {
if (!isStopped() && cache.available() == source.length()) {
cache.complete();
}
}

View File

@@ -1,13 +1,16 @@
package com.danikula.videocache;
import android.text.TextUtils;
import android.util.Log;
import android.webkit.MimeTypeMap;
import java.io.File;
import java.io.Closeable;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;
import static com.danikula.videocache.Preconditions.checkArgument;
@@ -18,7 +21,7 @@ import static com.danikula.videocache.Preconditions.checkNotNull;
*
* @author Alexey Danilov (danikula@gmail.com).
*/
class ProxyCacheUtils {
public class ProxyCacheUtils {
static final String LOG_TAG = "ProxyCache";
static final int DEFAULT_BUFFER_SIZE = 8 * 1024;
@@ -46,19 +49,6 @@ class ProxyCacheUtils {
return preview;
}
static void createDirectory(File directory) throws IOException {
checkNotNull(directory, "File must be not null!");
if (directory.exists()) {
checkArgument(directory.isDirectory(), "File is not directory!");
} else {
boolean isCreated = directory.mkdirs();
if (!isCreated) {
String error = String.format("Directory %s can't be created", directory.getAbsolutePath());
throw new IOException(error);
}
}
}
static String encode(String url) {
try {
return URLEncoder.encode(url, "utf-8");
@@ -74,4 +64,32 @@ class ProxyCacheUtils {
throw new RuntimeException("Error decoding url", e);
}
}
static void close(Closeable closeable) {
if (closeable != null) {
try {
closeable.close();
} catch (IOException e) {
Log.e(LOG_TAG, "Error closing resource", e);
}
}
}
public static String computeMD5(String string) {
try {
MessageDigest messageDigest = MessageDigest.getInstance("MD5");
byte[] digestBytes = messageDigest.digest(string.getBytes());
return bytesToHexString(digestBytes);
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException(e);
}
}
private static String bytesToHexString(byte[] bytes) {
StringBuffer sb = new StringBuffer();
for (byte b : bytes) {
sb.append(String.format("%02x", b));
}
return sb.toString();
}
}

View File

@@ -7,11 +7,34 @@ package com.danikula.videocache;
*/
public interface Source {
int available() throws ProxyCacheException;
/**
* Opens source. Source should be open before using {@link #read(byte[])}
*
* @param offset offset in bytes for source.
* @throws ProxyCacheException if error occur while opening source.
*/
void open(int offset) throws ProxyCacheException;
void close() throws ProxyCacheException;
/**
* Returns length bytes or <b>negative value</b> if length is unknown.
*
* @return bytes length
* @throws ProxyCacheException if error occur while fetching source data.
*/
int length() throws ProxyCacheException;
/**
* Read data to byte buffer from source with current offset.
*
* @param buffer a buffer to be used for reading data.
* @throws ProxyCacheException if error occur while reading source.
*/
int read(byte[] buffer) throws ProxyCacheException;
/**
* Closes source and release resources. Every opened source should be closed.
*
* @throws ProxyCacheException if error occur while closing source.
*/
void close() throws ProxyCacheException;
}

View File

@@ -0,0 +1,81 @@
package com.danikula.videocache;
import android.content.Context;
import android.os.Environment;
import android.util.Log;
import java.io.File;
import static android.os.Environment.MEDIA_MOUNTED;
import static com.danikula.videocache.ProxyCacheUtils.LOG_TAG;
/**
* Provides application storage paths
* <p/>
* See https://github.com/nostra13/Android-Universal-Image-Loader
*
* @author Sergey Tarasevich (nostra13[at]gmail[dot]com)
* @since 1.0.0
*/
final class StorageUtils {
private static final String INDIVIDUAL_DIR_NAME = "video-cache";
/**
* Returns individual application cache directory (for only video caching from Proxy). Cache directory will be
* created on SD card <i>("/Android/data/[app_package_name]/cache/video-cache")</i> if card is mounted .
* Else - Android defines cache directory on device's file system.
*
* @param context Application context
* @return Cache {@link File directory}
*/
public static File getIndividualCacheDirectory(Context context) {
File cacheDir = getCacheDirectory(context, true);
return new File(cacheDir, INDIVIDUAL_DIR_NAME);
}
/**
* Returns application cache directory. Cache directory will be created on SD card
* <i>("/Android/data/[app_package_name]/cache")</i> (if card is mounted and app has appropriate permission) or
* on device's file system depending incoming parameters.
*
* @param context Application context
* @param preferExternal Whether prefer external location for cache
* @return Cache {@link File directory}.<br />
* <b>NOTE:</b> Can be null in some unpredictable cases (if SD card is unmounted and
* {@link android.content.Context#getCacheDir() Context.getCacheDir()} returns null).
*/
private static File getCacheDirectory(Context context, boolean preferExternal) {
File appCacheDir = null;
String externalStorageState;
try {
externalStorageState = Environment.getExternalStorageState();
} catch (NullPointerException e) { // (sh)it happens
externalStorageState = "";
}
if (preferExternal && MEDIA_MOUNTED.equals(externalStorageState)) {
appCacheDir = getExternalCacheDir(context);
}
if (appCacheDir == null) {
appCacheDir = context.getCacheDir();
}
if (appCacheDir == null) {
String cacheDirPath = "/data/data/" + context.getPackageName() + "/cache/";
Log.w(LOG_TAG, "Can't define system cache directory! '" + cacheDirPath + "%s' will be used.");
appCacheDir = new File(cacheDirPath);
}
return appCacheDir;
}
private static File getExternalCacheDir(Context context) {
File dataDir = new File(new File(Environment.getExternalStorageDirectory(), "Android"), "data");
File appCacheDir = new File(new File(dataDir, context.getPackageName()), "cache");
if (!appCacheDir.exists()) {
if (!appCacheDir.mkdirs()) {
Log.w(LOG_TAG, "Unable to create external cache directory");
return null;
}
}
return appCacheDir;
}
}

View File

@@ -0,0 +1,15 @@
package com.danikula.videocache.file;
import java.io.File;
import java.io.IOException;
/**
* Declares how {@link FileCache} will use disc space.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
public interface DiskUsage {
void touch(File file) throws IOException;
}

View File

@@ -1,11 +1,12 @@
package com.danikula.videocache;
package com.danikula.videocache.file;
import com.danikula.videocache.Cache;
import com.danikula.videocache.ProxyCacheException;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import static com.danikula.videocache.Preconditions.checkNotNull;
/**
* {@link Cache} that uses file for storing data.
*
@@ -15,13 +16,22 @@ public class FileCache implements Cache {
private static final String TEMP_POSTFIX = ".download";
private final DiskUsage diskUsage;
public File file;
private RandomAccessFile dataFile;
public FileCache(File file) throws ProxyCacheException {
this(file, new UnlimitedDiskUsage());
}
public FileCache(File file, DiskUsage diskUsage) throws ProxyCacheException {
try {
checkNotNull(file);
ProxyCacheUtils.createDirectory(file.getParentFile());
if (diskUsage == null) {
throw new NullPointerException();
}
this.diskUsage = diskUsage;
File directory = file.getParentFile();
Files.makeDir(directory);
boolean completed = file.exists();
this.file = completed ? file : new File(file.getParentFile(), file.getName() + TEMP_POSTFIX);
this.dataFile = new RandomAccessFile(this.file, completed ? "r" : "rw");
@@ -68,6 +78,7 @@ public class FileCache implements Cache {
public synchronized void close() throws ProxyCacheException {
try {
dataFile.close();
diskUsage.touch(file);
} catch (IOException e) {
throw new ProxyCacheException("Error closing file " + file, e);
}

View File

@@ -1,6 +1,4 @@
package com.danikula.videocache;
import java.io.File;
package com.danikula.videocache.file;
/**
* Generator for files to be used for caching.
@@ -9,6 +7,6 @@ import java.io.File;
*/
public interface FileNameGenerator {
File generate(String url);
String generate(String url);
}

View File

@@ -0,0 +1,88 @@
package com.danikula.videocache.file;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.LinkedList;
import java.util.List;
/**
* Utils for work with files.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
class Files {
static void makeDir(File directory) throws IOException {
if (directory.exists()) {
if (!directory.isDirectory()) {
throw new IOException("File " + directory + " is not directory!");
}
} else {
boolean isCreated = directory.mkdirs();
if (!isCreated) {
throw new IOException(String.format("Directory %s can't be created", directory.getAbsolutePath()));
}
}
}
static List<File> getLruListFiles(File directory) {
List<File> result = new LinkedList<>();
File[] files = directory.listFiles();
if (files != null) {
result = Arrays.asList(files);
Collections.sort(result, new LastModifiedComparator());
}
return result;
}
static void setLastModifiedNow(File file) throws IOException {
if (file.exists()) {
long now = System.currentTimeMillis();
boolean modified = file.setLastModified(now); // on some devices (e.g. Nexus 5) doesn't work
if (!modified) {
modify(file);
if (file.lastModified() < now) {
throw new IOException("Error set last modified date to " + file);
}
}
}
}
static void modify(File file) throws IOException {
long size = file.length();
if (size == 0) {
recreateZeroSizeFile(file);
return;
}
RandomAccessFile accessFile = new RandomAccessFile(file, "rwd");
accessFile.seek(size - 1);
byte lastByte = accessFile.readByte();
accessFile.seek(size - 1);
accessFile.write(lastByte);
accessFile.close();
}
private static void recreateZeroSizeFile(File file) throws IOException {
if (!file.delete() || !file.createNewFile()) {
throw new IOException("Error recreate zero-size file " + file);
}
}
private static final class LastModifiedComparator implements Comparator<File> {
@Override
public int compare(File lhs, File rhs) {
return compareLong(lhs.lastModified(), rhs.lastModified());
}
private int compareLong(long first, long second) {
return (first < second) ? -1 : ((first == second) ? 0 : 1);
}
}
}

View File

@@ -0,0 +1,76 @@
package com.danikula.videocache.file;
import android.util.Log;
import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
/**
* {@link DiskUsage} that uses LRU (Least Recently Used) strategy to trim cache.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
abstract class LruDiskUsage implements DiskUsage {
private static final String LOG_TAG = "ProxyCache";
private final ExecutorService workerThread = Executors.newSingleThreadExecutor();
@Override
public void touch(File file) throws IOException {
workerThread.submit(new TouchCallable(file));
}
private void touchInBackground(File file) throws IOException {
Files.setLastModifiedNow(file);
List<File> files = Files.getLruListFiles(file.getParentFile());
trim(files);
}
protected abstract boolean accept(File file, long totalSize, int totalCount);
private void trim(List<File> files) {
long totalSize = countTotalSize(files);
int totalCount = files.size();
for (File file : files) {
boolean accepted = accept(file, totalSize, totalCount);
if (!accepted) {
long fileSize = file.length();
boolean deleted = file.delete();
if (deleted) {
totalCount--;
totalSize -= fileSize;
Log.i(LOG_TAG, "Cache file " + file + " is deleted because it exceeds cache limit");
} else {
Log.e(LOG_TAG, "Error deleting file " + file + " for trimming cache");
}
}
}
}
private long countTotalSize(List<File> files) {
long totalSize = 0;
for (File file : files) {
totalSize += file.length();
}
return totalSize;
}
private class TouchCallable implements Callable<Void> {
private final File file;
public TouchCallable(File file) {
this.file = file;
}
@Override
public Void call() throws Exception {
touchInBackground(file);
return null;
}
}
}

View File

@@ -0,0 +1,29 @@
package com.danikula.videocache.file;
import android.text.TextUtils;
import com.danikula.videocache.ProxyCacheUtils;
/**
* Implementation of {@link FileNameGenerator} that uses MD5 of url as file name
*
* @author Alexey Danilov (danikula@gmail.com).
*/
public class Md5FileNameGenerator implements FileNameGenerator {
private static final int MAX_EXTENSION_LENGTH = 4;
@Override
public String generate(String url) {
String extension = getExtension(url);
String name = ProxyCacheUtils.computeMD5(url);
return TextUtils.isEmpty(extension) ? name : name + "." + extension;
}
private String getExtension(String url) {
int dotIndex = url.lastIndexOf('.');
int slashIndex = url.lastIndexOf('/');
return dotIndex != -1 && dotIndex > slashIndex && dotIndex + 2 + MAX_EXTENSION_LENGTH > url.length() ?
url.substring(dotIndex + 1, url.length()) : "";
}
}

View File

@@ -0,0 +1,25 @@
package com.danikula.videocache.file;
import java.io.File;
/**
* {@link DiskUsage} that uses LRU (Least Recently Used) strategy and trims cache size to max files count if needed.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
public class TotalCountLruDiskUsage extends LruDiskUsage {
private final int maxCount;
public TotalCountLruDiskUsage(int maxCount) {
if (maxCount <= 0) {
throw new IllegalArgumentException("Max count must be positive number!");
}
this.maxCount = maxCount;
}
@Override
protected boolean accept(File file, long totalSize, int totalCount) {
return totalCount <= maxCount;
}
}

View File

@@ -0,0 +1,25 @@
package com.danikula.videocache.file;
import java.io.File;
/**
* {@link DiskUsage} that uses LRU (Least Recently Used) strategy and trims cache size to max size if needed.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
public class TotalSizeLruDiskUsage extends LruDiskUsage {
private final long maxSize;
public TotalSizeLruDiskUsage(long maxSize) {
if (maxSize <= 0) {
throw new IllegalArgumentException("Max size must be positive number!");
}
this.maxSize = maxSize;
}
@Override
protected boolean accept(File file, long totalSize, int totalCount) {
return totalSize <= maxSize;
}
}

View File

@@ -0,0 +1,17 @@
package com.danikula.videocache.file;
import java.io.File;
import java.io.IOException;
/**
* Unlimited version of {@link DiskUsage}.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
public class UnlimitedDiskUsage implements DiskUsage {
@Override
public void touch(File file) throws IOException {
// do nothing
}
}

View File

@@ -8,7 +8,6 @@ buildscript {
}
repositories {
maven { url 'https://dl.bintray.com/alexeydanilov/maven' }
maven { url 'https://github.com/dahlgren/vpi-aar/raw/master' }
}
@@ -17,7 +16,7 @@ apply plugin: 'com.neenbedankt.android-apt'
android {
compileSdkVersion 23
buildToolsVersion '23.0.0'
buildToolsVersion '23.0.1'
defaultConfig {
applicationId "com.danikula.videocache.sample"
@@ -37,9 +36,9 @@ apt {
dependencies {
// compile project(':library')
compile 'com.android.support:support-v4:23.0.0'
compile 'com.android.support:support-v4:23.1.0'
compile 'org.androidannotations:androidannotations-api:3.3.2'
compile 'com.danikula:videocache:2.0.7'
compile 'com.danikula:videocache:2.3.3'
compile 'com.viewpagerindicator:library:2.4.2-SNAPSHOT@aar'
apt 'org.androidannotations:androidannotations:3.3.2'
}

View File

@@ -3,9 +3,7 @@ package com.danikula.videocache.sample;
import android.app.Application;
import android.content.Context;
import com.danikula.videocache.FileNameGenerator;
import com.danikula.videocache.HttpProxyCacheServer;
import com.danikula.videocache.Md5FileNameGenerator;
/**
* @author Alexey Danilov (danikula@gmail.com).
@@ -20,7 +18,6 @@ public class App extends Application {
}
private HttpProxyCacheServer newProxy() {
FileNameGenerator nameGenerator = new Md5FileNameGenerator(getExternalCacheDir());
return new HttpProxyCacheServer(nameGenerator);
return new HttpProxyCacheServer(this);
}
}

View File

@@ -100,6 +100,7 @@ public class GalleryVideoFragment extends Fragment implements CacheListener {
public void onDestroy() {
super.onDestroy();
videoView.stopPlayback();
App.getProxy(getActivity()).unregisterCacheListener(this);
}

View File

@@ -3,9 +3,11 @@ package com.danikula.videocache.sample;
import android.content.Intent;
import android.support.annotation.NonNull;
import android.support.v4.app.FragmentActivity;
import android.util.Log;
import android.widget.ArrayAdapter;
import android.widget.ListAdapter;
import android.widget.ListView;
import android.widget.Toast;
import org.androidannotations.annotations.AfterViews;
import org.androidannotations.annotations.Click;
@@ -13,7 +15,7 @@ import org.androidannotations.annotations.EActivity;
import org.androidannotations.annotations.ItemClick;
import org.androidannotations.annotations.ViewById;
import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
@@ -46,11 +48,11 @@ public class MenuActivity extends FragmentActivity {
@Click(R.id.cleanCacheButton)
void onClearCacheButtonClick() {
File externalCacheDir = getExternalCacheDir();
if (externalCacheDir != null) {
for (File cacheEntry : externalCacheDir.listFiles()) {
cacheEntry.delete();
}
try {
Utils.cleanDirectory(getExternalCacheDir());
} catch (IOException e) {
Log.e(null, "Error cleaning cache", e);
Toast.makeText(this, "Error cleaning cache", Toast.LENGTH_LONG).show();
}
}

View File

@@ -0,0 +1,42 @@
package com.danikula.videocache.sample;
import java.io.File;
import java.io.IOException;
/**
* Some utils methods.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
public class Utils {
public static void cleanDirectory(File file) throws IOException {
if (!file.exists()) {
return;
}
File[] contentFiles = file.listFiles();
if (contentFiles != null) {
for (File contentFile : contentFiles) {
delete(contentFile);
}
}
}
private static void delete(File file) throws IOException {
if (file.isFile() && file.exists()) {
deleteOrThrow(file);
} else {
cleanDirectory(file);
deleteOrThrow(file);
}
}
private static void deleteOrThrow(File file) throws IOException {
if (file.exists()) {
boolean isDeleted = file.delete();
if (!isDeleted) {
throw new IOException(String.format("File %s can't be deleted", file.getAbsolutePath()));
}
}
}
}

View File

@@ -4,6 +4,7 @@ import android.content.Context;
import android.os.Handler;
import android.os.Message;
import android.support.v4.app.Fragment;
import android.util.Log;
import android.widget.ProgressBar;
import android.widget.VideoView;
@@ -70,12 +71,14 @@ public class VideoFragment extends Fragment implements CacheListener {
public void onDestroy() {
super.onDestroy();
videoView.stopPlayback();
App.getProxy(getActivity()).unregisterCacheListener(this);
}
@Override
public void onCacheAvailable(File file, String url, int percentsAvailable) {
progressBar.setSecondaryProgress(percentsAvailable);
Log.d(LOG_TAG, String.format("onCacheAvailable. percents: %d, file: %s, url: %s", percentsAvailable, file, url));
}
private void updateVideoProgress() {

View File

@@ -0,0 +1,79 @@
package com.danikula.videocache;
import com.danikula.videocache.file.FileNameGenerator;
import com.danikula.videocache.file.Md5FileNameGenerator;
import com.danikula.videocache.test.BuildConfig;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.robolectric.RobolectricGradleTestRunner;
import org.robolectric.annotation.Config;
import java.io.File;
import static org.fest.assertions.api.Assertions.assertThat;
import static org.fest.assertions.api.Assertions.fail;
/**
* Tests for {@link FileNameGenerator} and implementations.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
@RunWith(RobolectricGradleTestRunner.class)
@Config(constants = BuildConfig.class, emulateSdk = BuildConfig.MIN_SDK_VERSION)
public class FileNameGeneratorTest {
@Test
public void testMd5SimpleName() throws Exception {
String url = "http://host.com/videos/video.mpeg";
String path = generateMd5Name("/home", url);
String expected = "/home/" + ProxyCacheUtils.computeMD5(url) + ".mpeg";
assertThat(path).isEqualTo(expected);
}
@Test
public void testMd5NoExtension() throws Exception {
String url = "http://host.com/video";
String path = generateMd5Name("/home", url);
String expected = "/home/" + ProxyCacheUtils.computeMD5(url);
assertThat(path).isEqualTo(expected);
}
@Test
public void testMd5TooLongExtension() throws Exception {
String url = "http://host.com/videos/video-with-dot-.12345";
String path = generateMd5Name("/home", url);
String expected = "/home/" + ProxyCacheUtils.computeMD5(url);
assertThat(path).isEqualTo(expected);
}
@Test
public void testMd5InvalidExtension() throws Exception {
String url = "http://host.com/videos/video.mp4?token=-648729473536183645";
String path = generateMd5Name("/home", url);
String expected = "/home/" + ProxyCacheUtils.computeMD5(url);
assertThat(path).isEqualTo(expected);
}
@Test
public void testMd5ExtraLongExtension() throws Exception {
// https://github.com/danikula/AndroidVideoCache/issues/14
String url = "https://d1wst0behutosd.cloudfront.net/videos/4367900/10807247.480p.mp4?Expires=1442849176&Signature=JXV~3AoI0rWcGuZBywg3-ukf6Ycw2X8v7Htog3lyvuFwp8o6VUEDFUsTC9-XtIGu-ULxCd7dP3fvB306lRyGFxdvf-sXLX~ar~HCQ7lullNyeLtp8BJOT5Y~W5rJE7X-AZaueNcycGtLFRhRtr5ySTguwtmJNaO3T1apX~-oVrFh1dWStEKbuPoXY04RgkmhMHoFgtwgXMC1ctIDeQHxZeXLi6LLyZnQsgzlUDffCx4P16iiW0uh2-Z~HUOi9BLBwHMQ5k5lYwZqdQ6DhhYoWlniRfQz6mp1IEiMgr4L3Z1ijgGITV4cYeF31CmFzCxaJTE7IIAC5tMDQSTt7M9Q4A__&Key-Pair-Id=APKAJJ6WELAPEP47UKWQ";
String path = generateMd5Name("/home", url);
String expected = "/home/" + ProxyCacheUtils.computeMD5(url);
assertThat(path).isEqualTo(expected);
}
@Test(expected = NullPointerException.class)
public void testAssertNullUrl() throws Exception {
FileNameGenerator nameGenerator = new Md5FileNameGenerator();
nameGenerator.generate(null);
fail("Url should be not null");
}
private String generateMd5Name(String rootFolder, String url) {
FileNameGenerator nameGenerator = new Md5FileNameGenerator();
String name = nameGenerator.generate(url);
return new File(rootFolder, name).getAbsolutePath();
}
}

View File

@@ -3,10 +3,13 @@ package com.danikula.videocache;
import android.util.Pair;
import com.danikula.android.garden.io.IoUtils;
import com.danikula.videocache.file.FileNameGenerator;
import com.danikula.videocache.file.Md5FileNameGenerator;
import com.danikula.videocache.support.ProxyCacheTestUtils;
import com.danikula.videocache.support.Response;
import com.danikula.videocache.test.BuildConfig;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.robolectric.RobolectricGradleTestRunner;
@@ -17,11 +20,18 @@ import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import static com.danikula.android.garden.io.Files.cleanDirectory;
import static com.danikula.android.garden.io.Files.createDirectory;
import static com.danikula.videocache.support.ProxyCacheTestUtils.ASSETS_DATA_BIG_NAME;
import static com.danikula.videocache.support.ProxyCacheTestUtils.ASSETS_DATA_NAME;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_BIG_SIZE;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_BIG_URL;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_BIG_URL_ONE_REDIRECT;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_SIZE;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL_3_REDIRECTS;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL_6_REDIRECTS;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL_ONE_REDIRECT;
import static com.danikula.videocache.support.ProxyCacheTestUtils.getFileContent;
import static com.danikula.videocache.support.ProxyCacheTestUtils.loadAssetFile;
import static com.danikula.videocache.support.ProxyCacheTestUtils.readProxyResponse;
@@ -34,6 +44,15 @@ import static org.fest.assertions.api.Assertions.assertThat;
@Config(constants = BuildConfig.class, emulateSdk = BuildConfig.MIN_SDK_VERSION)
public class HttpProxyCacheServerTest {
private File cacheFolder;
@Before
public void setup() throws Exception {
cacheFolder = ProxyCacheTestUtils.newCacheFile();
createDirectory(cacheFolder);
cleanDirectory(cacheFolder);
}
@Test
public void testHttpProxyCache() throws Exception {
Pair<File, Response> response = readProxyData(HTTP_DATA_URL);
@@ -45,14 +64,14 @@ public class HttpProxyCacheServerTest {
@Test
public void testProxyContentWithPartialCache() throws Exception {
FileNameGenerator fileNameGenerator = new Md5FileNameGenerator(RuntimeEnvironment.application.getExternalCacheDir());
File file = fileNameGenerator.generate(HTTP_DATA_URL);
File cacheDir = RuntimeEnvironment.application.getExternalCacheDir();
File file = new File(cacheDir, new Md5FileNameGenerator().generate(HTTP_DATA_URL));
int partialCacheSize = 1000;
byte[] partialData = ProxyCacheTestUtils.generate(partialCacheSize);
File partialCacheFile = ProxyCacheTestUtils.getTempFile(file);
IoUtils.saveToFile(partialData, partialCacheFile);
HttpProxyCacheServer proxy = new HttpProxyCacheServer(fileNameGenerator);
HttpProxyCacheServer proxy = newProxy(cacheDir);
Response response = readProxyResponse(proxy, HTTP_DATA_URL);
proxy.shutdown();
@@ -81,9 +100,23 @@ public class HttpProxyCacheServerTest {
assertThat(response.second.data).isEqualTo(loadAssetFile(ASSETS_DATA_BIG_NAME));
}
@Test
public void testProxyFullResponseWithRedirect() throws Exception {
Pair<File, Response> response = readProxyData(HTTP_DATA_BIG_URL_ONE_REDIRECT);
assertThat(response.second.code).isEqualTo(200);
assertThat(response.second.contentLength).isEqualTo(HTTP_DATA_BIG_SIZE);
assertThat(response.second.contentType).isEqualTo("image/jpeg");
assertThat(response.second.headers.containsKey("Accept-Ranges")).isTrue();
assertThat(response.second.headers.get("Accept-Ranges").get(0)).isEqualTo("bytes");
assertThat(response.second.headers.containsKey("Content-Range")).isFalse();
assertThat(response.second.data).isEqualTo(getFileContent(response.first));
assertThat(response.second.data).isEqualTo(loadAssetFile(ASSETS_DATA_BIG_NAME));
}
@Test
public void testProxyPartialResponse() throws Exception {
int offset = 42000;
int offset = 18000;
Pair<File, Response> response = readProxyData(HTTP_DATA_BIG_URL, offset);
assertThat(response.second.code).isEqualTo(206);
@@ -92,18 +125,94 @@ public class HttpProxyCacheServerTest {
assertThat(response.second.headers.containsKey("Accept-Ranges")).isTrue();
assertThat(response.second.headers.get("Accept-Ranges").get(0)).isEqualTo("bytes");
assertThat(response.second.headers.containsKey("Content-Range")).isTrue();
String rangeHeader = String.format("bytes %d-%d/%d", offset, HTTP_DATA_BIG_SIZE, HTTP_DATA_BIG_SIZE);
String rangeHeader = String.format("bytes %d-%d/%d", offset, HTTP_DATA_BIG_SIZE - 1, HTTP_DATA_BIG_SIZE);
assertThat(response.second.headers.get("Content-Range").get(0)).isEqualTo(rangeHeader);
byte[] expectedData = Arrays.copyOfRange(loadAssetFile(ASSETS_DATA_BIG_NAME), offset, HTTP_DATA_BIG_SIZE);
assertThat(response.second.data).isEqualTo(expectedData);
assertThat(getFileContent(response.first)).isEqualTo(loadAssetFile(ASSETS_DATA_BIG_NAME));
}
@Test
public void testProxyPartialResponseWithRedirect() throws Exception {
int offset = 18000;
Pair<File, Response> response = readProxyData(HTTP_DATA_BIG_URL_ONE_REDIRECT, offset);
assertThat(response.second.code).isEqualTo(206);
assertThat(response.second.contentLength).isEqualTo(HTTP_DATA_BIG_SIZE - offset);
assertThat(response.second.contentType).isEqualTo("image/jpeg");
assertThat(response.second.headers.containsKey("Accept-Ranges")).isTrue();
assertThat(response.second.headers.get("Accept-Ranges").get(0)).isEqualTo("bytes");
assertThat(response.second.headers.containsKey("Content-Range")).isTrue();
String rangeHeader = String.format("bytes %d-%d/%d", offset, HTTP_DATA_BIG_SIZE - 1, HTTP_DATA_BIG_SIZE);
assertThat(response.second.headers.get("Content-Range").get(0)).isEqualTo(rangeHeader);
byte[] expectedData = Arrays.copyOfRange(loadAssetFile(ASSETS_DATA_BIG_NAME), offset, HTTP_DATA_BIG_SIZE);
assertThat(response.second.data).isEqualTo(expectedData);
assertThat(getFileContent(response.first)).isEqualTo(loadAssetFile(ASSETS_DATA_BIG_NAME));
}
@Test
public void testMaxSizeCacheLimit() throws Exception {
HttpProxyCacheServer proxy = new HttpProxyCacheServer.Builder(RuntimeEnvironment.application)
.cacheDirectory(cacheFolder)
.maxCacheSize(HTTP_DATA_SIZE * 3 - 1) // for 2 files
.build();
// use different url (doesn't matter than same content)
readProxyResponse(proxy, HTTP_DATA_URL, 0);
Thread.sleep(1050); // wait for new last modified date (file rounds time to second)
readProxyResponse(proxy, HTTP_DATA_URL_ONE_REDIRECT, 0);
Thread.sleep(1050);
readProxyResponse(proxy, HTTP_DATA_URL_3_REDIRECTS, 0);
Thread.sleep(1050);
assertThat(file(cacheFolder, HTTP_DATA_URL)).doesNotExist();
assertThat(file(cacheFolder, HTTP_DATA_URL_ONE_REDIRECT)).exists();
assertThat(file(cacheFolder, HTTP_DATA_URL_3_REDIRECTS)).exists();
readProxyResponse(proxy, HTTP_DATA_URL_ONE_REDIRECT, 0); // touch file
readProxyResponse(proxy, HTTP_DATA_URL_6_REDIRECTS, 0);
proxy.shutdown();
assertThat(file(cacheFolder, HTTP_DATA_URL_3_REDIRECTS)).doesNotExist();
assertThat(file(cacheFolder, HTTP_DATA_URL_ONE_REDIRECT)).exists();
assertThat(file(cacheFolder, HTTP_DATA_URL_6_REDIRECTS)).exists();
}
@Test
public void testMaxFileCacheLimit() throws Exception {
HttpProxyCacheServer proxy = new HttpProxyCacheServer.Builder(RuntimeEnvironment.application)
.cacheDirectory(cacheFolder)
.maxCacheFilesCount(2)
.build();
// use different url (doesn't matter than same content)
readProxyResponse(proxy, HTTP_DATA_URL, 0);
Thread.sleep(1050); // wait for new last modified date (file rounds time to second)
readProxyResponse(proxy, HTTP_DATA_URL_ONE_REDIRECT, 0);
Thread.sleep(1050);
readProxyResponse(proxy, HTTP_DATA_URL_3_REDIRECTS, 0);
Thread.sleep(1050);
assertThat(file(cacheFolder, HTTP_DATA_URL)).doesNotExist();
assertThat(file(cacheFolder, HTTP_DATA_URL_ONE_REDIRECT)).exists();
assertThat(file(cacheFolder, HTTP_DATA_URL_3_REDIRECTS)).exists();
readProxyResponse(proxy, HTTP_DATA_URL_ONE_REDIRECT, 0); // touch file
readProxyResponse(proxy, HTTP_DATA_URL_6_REDIRECTS, 0);
proxy.shutdown();
assertThat(file(cacheFolder, HTTP_DATA_URL_3_REDIRECTS)).doesNotExist();
assertThat(file(cacheFolder, HTTP_DATA_URL_ONE_REDIRECT)).exists();
assertThat(file(cacheFolder, HTTP_DATA_URL_6_REDIRECTS)).exists();
}
private Pair<File, Response> readProxyData(String url, int offset) throws IOException {
File externalCacheDir = RuntimeEnvironment.application.getExternalCacheDir();
FileNameGenerator fileNameGenerator = new Md5FileNameGenerator(externalCacheDir);
File file = fileNameGenerator.generate(url);
HttpProxyCacheServer proxy = new HttpProxyCacheServer(fileNameGenerator);
File file = file(cacheFolder, url);
HttpProxyCacheServer proxy = newProxy(cacheFolder);
Response response = readProxyResponse(proxy, url, offset);
proxy.shutdown();
@@ -111,7 +220,19 @@ public class HttpProxyCacheServerTest {
return new Pair<>(file, response);
}
private File file(File parent, String url) {
FileNameGenerator fileNameGenerator = new Md5FileNameGenerator();
String name = fileNameGenerator.generate(url);
return new File(parent, name);
}
private Pair<File, Response> readProxyData(String url) throws IOException {
return readProxyData(url, -1);
}
private HttpProxyCacheServer newProxy(File cacheDir) {
return new HttpProxyCacheServer.Builder(RuntimeEnvironment.application)
.cacheDirectory(cacheDir)
.build();
}
}

View File

@@ -0,0 +1,98 @@
package com.danikula.videocache;
import com.danikula.videocache.file.FileCache;
import com.danikula.videocache.support.ProxyCacheTestUtils;
import com.danikula.videocache.support.Response;
import com.danikula.videocache.test.BuildConfig;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;
import org.robolectric.RobolectricGradleTestRunner;
import org.robolectric.annotation.Config;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.net.Socket;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL;
import static com.danikula.videocache.support.ProxyCacheTestUtils.loadTestData;
import static org.fest.assertions.api.Assertions.assertThat;
import static org.mockito.Matchers.any;
import static org.mockito.Matchers.anyInt;
import static org.mockito.Matchers.anyLong;
import static org.mockito.Matchers.eq;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* Test {@link HttpProxyCache}.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
@RunWith(RobolectricGradleTestRunner.class)
@Config(constants = BuildConfig.class, emulateSdk = BuildConfig.MIN_SDK_VERSION)
public class HttpProxyCacheTest {
@Test
public void testProcessRequestNoCache() throws Exception {
HttpUrlSource source = new HttpUrlSource(HTTP_DATA_URL);
FileCache cache = new FileCache(ProxyCacheTestUtils.newCacheFile());
HttpProxyCache proxyCache = new HttpProxyCache(source, cache);
GetRequest request = new GetRequest("GET /" + HTTP_DATA_URL + " HTTP/1.1");
ByteArrayOutputStream out = new ByteArrayOutputStream();
Socket socket = mock(Socket.class);
when(socket.getOutputStream()).thenReturn(out);
proxyCache.processRequest(request, socket);
Response response = new Response(out.toByteArray());
assertThat(response.data).isEqualTo(loadTestData());
assertThat(response.code).isEqualTo(200);
assertThat(response.contentLength).isEqualTo(ProxyCacheTestUtils.HTTP_DATA_SIZE);
assertThat(response.contentType).isEqualTo("image/jpeg");
}
@Test
public void testProcessPartialRequestWithoutCache() throws Exception {
HttpUrlSource source = new HttpUrlSource(HTTP_DATA_URL);
FileCache fileCache = new FileCache(ProxyCacheTestUtils.newCacheFile());
FileCache spyFileCache = Mockito.spy(fileCache);
doThrow(new RuntimeException()).when(spyFileCache).read(any(byte[].class), anyLong(), anyInt());
HttpProxyCache proxyCache = new HttpProxyCache(source, spyFileCache);
GetRequest request = new GetRequest("GET /" + HTTP_DATA_URL + " HTTP/1.1\nRange: bytes=2000-");
ByteArrayOutputStream out = new ByteArrayOutputStream();
Socket socket = mock(Socket.class);
when(socket.getOutputStream()).thenReturn(out);
proxyCache.processRequest(request, socket);
Response response = new Response(out.toByteArray());
byte[] fullData = loadTestData();
byte[] partialData = new byte[fullData.length - 2000];
System.arraycopy(fullData, 2000, partialData, 0, partialData.length);
assertThat(response.data).isEqualTo(partialData);
assertThat(response.code).isEqualTo(206);
}
@Test
public void testLoadEmptyFile() throws Exception {
String zeroSizeUrl = "https://dl.dropboxusercontent.com/u/15506779/persistent/proxycache/empty.txt";
HttpUrlSource source = new HttpUrlSource(zeroSizeUrl);
HttpProxyCache proxyCache = new HttpProxyCache(source, new FileCache(ProxyCacheTestUtils.newCacheFile()));
GetRequest request = new GetRequest("GET /" + HTTP_DATA_URL + " HTTP/1.1");
ByteArrayOutputStream out = new ByteArrayOutputStream();
Socket socket = mock(Socket.class);
when(socket.getOutputStream()).thenReturn(out);
CacheListener listener = Mockito.mock(CacheListener.class);
proxyCache.registerCacheListener(listener);
proxyCache.processRequest(request, socket);
proxyCache.registerCacheListener(null);
Response response = new Response(out.toByteArray());
Mockito.verify(listener).onCacheAvailable(Mockito.<File>any(), eq(zeroSizeUrl), eq(100));
assertThat(response.data).isEmpty();
}
}

View File

@@ -15,9 +15,14 @@ import static com.danikula.videocache.support.ProxyCacheTestUtils.ASSETS_DATA_BI
import static com.danikula.videocache.support.ProxyCacheTestUtils.ASSETS_DATA_NAME;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_BIG_SIZE;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_BIG_URL;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_SIZE;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL_3_REDIRECTS;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL_6_REDIRECTS;
import static com.danikula.videocache.support.ProxyCacheTestUtils.HTTP_DATA_URL_ONE_REDIRECT;
import static com.danikula.videocache.support.ProxyCacheTestUtils.loadAssetFile;
import static org.fest.assertions.api.Assertions.assertThat;
import static org.fest.assertions.api.Assertions.fail;
/**
* @author Alexey Danilov (danikula@gmail.com).
@@ -58,7 +63,65 @@ public class HttpUrlSourceTest {
@Test
public void testFetchContentLength() throws Exception {
Source source = new HttpUrlSource(HTTP_DATA_URL);
assertThat(source.available()).isEqualTo(loadAssetFile(ASSETS_DATA_NAME).length);
assertThat(source.length()).isEqualTo(loadAssetFile(ASSETS_DATA_NAME).length);
}
@Test
public void testFetchInfoWithRedirect() throws Exception {
HttpUrlSource source = new HttpUrlSource(HTTP_DATA_URL_ONE_REDIRECT);
source.open(0);
int available = source.length();
String mime = source.getMime();
source.close();
assertThat(available).isEqualTo(HTTP_DATA_SIZE);
assertThat(mime).isEqualTo("image/jpeg");
}
@Test
public void testFetchDataWithRedirect() throws Exception {
HttpUrlSource source = new HttpUrlSource(HTTP_DATA_URL_ONE_REDIRECT);
source.open(0);
byte[] readData = new byte[HTTP_DATA_SIZE];
source.read(readData);
source.close();
byte[] expectedData = Arrays.copyOfRange(loadAssetFile(ASSETS_DATA_NAME), 0, HTTP_DATA_SIZE);
assertThat(readData).isEqualTo(expectedData);
}
@Test
public void testFetchPartialDataWithRedirect() throws Exception {
int offset = 42;
HttpUrlSource source = new HttpUrlSource(HTTP_DATA_URL_ONE_REDIRECT);
source.open(offset);
byte[] readData = new byte[HTTP_DATA_SIZE - offset];
source.read(readData);
source.close();
byte[] expectedData = Arrays.copyOfRange(loadAssetFile(ASSETS_DATA_NAME), offset, HTTP_DATA_SIZE);
assertThat(readData).isEqualTo(expectedData);
}
@Test
public void testFetchPartialDataWithMultiRedirects() throws Exception {
int offset = 42;
HttpUrlSource source = new HttpUrlSource(HTTP_DATA_URL_3_REDIRECTS);
source.open(offset);
byte[] readData = new byte[HTTP_DATA_SIZE - offset];
source.read(readData);
source.close();
byte[] expectedData = Arrays.copyOfRange(loadAssetFile(ASSETS_DATA_NAME), offset, HTTP_DATA_SIZE);
assertThat(readData).isEqualTo(expectedData);
}
@Ignore("To test it fairly we should disable caching connection.setUseCaches(false), but it will decrease performance")
@Test(expected = ProxyCacheException.class)
public void testExceedingRedirects() throws Exception {
HttpUrlSource source = new HttpUrlSource(HTTP_DATA_URL_6_REDIRECTS);
source.open(0);
fail("Too many redirects");
}
@Ignore("Seems Robolectric bug: MimeTypeMap.getFileExtensionFromUrl always returns null")

View File

@@ -1,11 +1,11 @@
package com.danikula.videocache;
import com.danikula.android.garden.io.IoUtils;
import com.danikula.videocache.file.FileCache;
import com.danikula.videocache.support.AngryHttpUrlSource;
import com.danikula.videocache.support.PhlegmaticByteArraySource;
import com.danikula.videocache.test.BuildConfig;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.robolectric.RobolectricGradleTestRunner;
@@ -91,17 +91,6 @@ public class ProxyCacheTest {
assertThat(fetchedData).isEqualTo(sourceCopy);
}
@Test(expected = ProxyCacheException.class)
public void testNoMoreSource() throws Exception {
int sourceSize = 942;
int cacheSize = 6157;
ByteArraySource source = new ByteArraySource(generate(sourceSize));
ByteArrayCache cache = new ByteArrayCache(generate(cacheSize));
ProxyCache proxyCache = new ProxyCache(source, cache);
proxyCache.read(new byte[sourceSize + cacheSize], sourceSize + cacheSize + 1, 10);
Assert.fail();
}
@Test
public void testProxyWithPhlegmaticSource() throws Exception {
int dataSize = 100000;

View File

@@ -0,0 +1,122 @@
package com.danikula.videocache.file;
import com.danikula.videocache.support.ProxyCacheTestUtils;
import com.danikula.videocache.test.BuildConfig;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.robolectric.RobolectricGradleTestRunner;
import org.robolectric.annotation.Config;
import java.io.File;
import java.io.IOException;
import static com.danikula.android.garden.io.Files.cleanDirectory;
import static com.danikula.android.garden.io.Files.createDirectory;
import static org.fest.assertions.api.Assertions.assertThat;
/**
* Tests for implementations of {@link DiskUsage}.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
@RunWith(RobolectricGradleTestRunner.class)
@Config(constants = BuildConfig.class, emulateSdk = BuildConfig.MIN_SDK_VERSION)
public class DiskUsageTest {
private File cacheFolder;
@Before
public void setup() throws Exception {
cacheFolder = ProxyCacheTestUtils.newCacheFile();
createDirectory(cacheFolder);
cleanDirectory(cacheFolder);
}
@Test
public void testMaxSizeCacheLimit() throws Exception {
DiskUsage diskUsage = new TotalSizeLruDiskUsage(300);
long now = System.currentTimeMillis();
createFile(file("b"), 101, now - 10000);
createFile(file("c"), 102, now - 8000);
createFile(file("a"), 104, now - 4000); // exceeds
diskUsage.touch(file("c"));
waitForAsyncTrimming();
assertThat(file("b")).doesNotExist();
assertThat(file("c")).exists();
assertThat(file("a")).exists();
createFile(file("d"), 500, now); // exceeds all
diskUsage.touch(file("d"));
waitForAsyncTrimming();
assertThat(file("a")).doesNotExist();
assertThat(file("c")).doesNotExist();
assertThat(file("d")).doesNotExist();
}
@Test
public void testMaxFilesCount() throws Exception {
DiskUsage diskUsage = new TotalCountLruDiskUsage(2);
long now = System.currentTimeMillis();
createFile(file("b"), 101, now - 10000);
createFile(file("c"), 102, now - 8000);
createFile(file("a"), 104, now - 4000);
diskUsage.touch(file("c"));
waitForAsyncTrimming();
assertThat(file("b")).doesNotExist();
assertThat(file("a")).exists();
assertThat(file("c")).exists();
createFile(file("d"), 500, now);
diskUsage.touch(file("d"));
waitForAsyncTrimming();
assertThat(file("a")).doesNotExist();
assertThat(file("c")).exists();
assertThat(file("d")).exists();
}
@Test
public void testTouch() throws Exception {
DiskUsage diskUsage = new TotalCountLruDiskUsage(2);
long now = System.currentTimeMillis();
createFile(file("b"), 101, now - 10000);
createFile(file("c"), 102, now - 8000);
createFile(file("a"), 104, now - 4000);
diskUsage.touch(file("b"));
waitForAsyncTrimming();
assertThat(file("b")).exists();
assertThat(file("a")).exists();
assertThat(file("c")).doesNotExist();
Thread.sleep(1000); // last modified is rounded to seconds, so wait for sec
new TotalCountLruDiskUsage(1).touch(file("a"));
waitForAsyncTrimming();
assertThat(file("a")).exists();
assertThat(file("b")).doesNotExist();
}
private void waitForAsyncTrimming() throws InterruptedException {
Thread.sleep(200);
}
private File file(String name) {
return new File(cacheFolder, name);
}
private void createFile(File file, int capacity, long lastModified) throws IOException {
byte[] data = ProxyCacheTestUtils.generate(capacity);
com.google.common.io.Files.write(data, file);
boolean modified = file.setLastModified(lastModified);
assertThat(modified).isTrue();
}
}

View File

@@ -1,7 +1,9 @@
package com.danikula.videocache;
package com.danikula.videocache.file;
import com.danikula.android.garden.io.Files;
import com.danikula.android.garden.io.IoUtils;
import com.danikula.videocache.Cache;
import com.danikula.videocache.ProxyCacheException;
import com.danikula.videocache.test.BuildConfig;
import org.junit.Assert;

View File

@@ -0,0 +1,62 @@
package com.danikula.videocache.file;
import com.danikula.videocache.support.ProxyCacheTestUtils;
import com.danikula.videocache.test.BuildConfig;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.robolectric.RobolectricGradleTestRunner;
import org.robolectric.annotation.Config;
import java.io.File;
import static org.fest.assertions.api.Assertions.assertThat;
/**
* Tests for {@link Files}.
*
* @author Alexey Danilov (danikula@gmail.com).
*/
@RunWith(RobolectricGradleTestRunner.class)
@Config(constants = BuildConfig.class, emulateSdk = BuildConfig.MIN_SDK_VERSION)
public class FilesTest {
@Test
public void testModify() throws Exception {
byte[] data = ProxyCacheTestUtils.generate(999);
File file = ProxyCacheTestUtils.newCacheFile();
com.google.common.io.Files.write(data, file);
long lastModified = file.lastModified();
Thread.sleep(1100); // file can store modification date in seconds. so wait for ~ 1 sec
Files.modify(file);
assertThat(file).hasBinaryContent(data);
assertThat(file.lastModified()).isGreaterThan(lastModified);
}
@Test
public void testSetModifiedNow() throws Exception {
File file = ProxyCacheTestUtils.newCacheFile();
com.google.common.io.Files.write(ProxyCacheTestUtils.generate(22), file);
Thread.sleep(1100); // file can store modification date in seconds. so wait for ~ 1 sec
long nowRoundedToSecond = System.currentTimeMillis() / 1000 * 1000;
Files.setLastModifiedNow(file);
assertThat(file.lastModified()).isGreaterThanOrEqualTo(nowRoundedToSecond);
}
@Test
public void testModifyZeroSizeFile() throws Exception {
File file = ProxyCacheTestUtils.newCacheFile();
boolean created = file.createNewFile();
assertThat(created).isTrue();
Thread.sleep(1100); // file can store modification date in seconds. so wait for ~ 2 sec
long nowRoundedToSecond = System.currentTimeMillis() / 1000 * 1000;
Files.modify(file);
assertThat(file.lastModified()).isGreaterThanOrEqualTo(nowRoundedToSecond);
}
}

View File

@@ -8,10 +8,11 @@ import com.danikula.videocache.Source;
*
* @author Alexey Danilov (danikula@gmail.com).
*/
@Deprecated // use Mockito to throw error
public class AngryHttpUrlSource implements Source {
@Override
public int available() throws ProxyCacheException {
public int length() throws ProxyCacheException {
throw new IllegalStateException();
}

View File

@@ -8,6 +8,7 @@ import java.util.Random;
/**
* @author Alexey Danilov (danikula@gmail.com).
*/
@Deprecated // TODO: use Mockito to mock delay
public class PhlegmaticByteArraySource extends ByteArraySource {
private final Random delayGenerator;

View File

@@ -21,7 +21,11 @@ import java.util.UUID;
public class ProxyCacheTestUtils {
public static final String HTTP_DATA_URL = "https://dl.dropboxusercontent.com/u/15506779/persistent/proxycache/android.jpg";
public static final String HTTP_DATA_URL_ONE_REDIRECT = "http://bit.ly/1V5PeY5";
public static final String HTTP_DATA_URL_3_REDIRECTS = "http://bit.ly/1KvVmgZ";
public static final String HTTP_DATA_URL_6_REDIRECTS = "http://ow.ly/SugRH";
public static final String HTTP_DATA_BIG_URL = "https://dl.dropboxusercontent.com/u/15506779/persistent/proxycache/phones.jpg";
public static final String HTTP_DATA_BIG_URL_ONE_REDIRECT = "http://bit.ly/1iJ69yA";
public static final String ASSETS_DATA_NAME = "android.jpg";
public static final String ASSETS_DATA_BIG_NAME = "phones.jpg";
public static final int HTTP_DATA_SIZE = 4768;
@@ -36,7 +40,11 @@ public class ProxyCacheTestUtils {
}
public static Response readProxyResponse(HttpProxyCacheServer proxy, String url, int offset) throws IOException {
URL proxiedUrl = new URL(proxy.getProxyUrl(url));
String proxyUrl = proxy.getProxyUrl(url);
if (!proxyUrl.startsWith("http://127.0.0.1")) {
throw new IllegalStateException("Url " + url + " is not proxied!");
}
URL proxiedUrl = new URL(proxyUrl);
HttpURLConnection connection = (HttpURLConnection) proxiedUrl.openConnection();
try {
if (offset >= 0) {
@@ -48,6 +56,10 @@ public class ProxyCacheTestUtils {
}
}
public static byte[] loadTestData() throws IOException {
return loadAssetFile(ASSETS_DATA_NAME);
}
public static byte[] loadAssetFile(String name) throws IOException {
InputStream in = RuntimeEnvironment.application.getResources().getAssets().open(name);
ByteArrayOutputStream out = new ByteArrayOutputStream();

View File

@@ -1,14 +1,27 @@
package com.danikula.videocache.support;
import android.text.TextUtils;
import com.google.common.base.Preconditions;
import com.google.common.io.ByteStreams;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.StringReader;
import java.net.HttpURLConnection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class Response {
private static final String CONTENT_TYPE_HEADER = "Content-Type";
private static final String CONTENT_LENGTH_HEADER = "Content-Length";
private static final Pattern STATUS_CODE_PATTERN = Pattern.compile("HTTP/1.1 (\\d{3}) ");
public final int code;
public final byte[] data;
public final int contentLength;
@@ -22,4 +35,33 @@ public class Response {
this.headers = connection.getHeaderFields();
this.data = ByteStreams.toByteArray(connection.getInputStream());
}
public Response(byte[] responseData) throws IOException {
int read = 0;
BufferedReader reader = new BufferedReader(new StringReader(new String(responseData, "ascii")));
String statusLine = reader.readLine();
read += statusLine.length() + 1;
Matcher matcher = STATUS_CODE_PATTERN.matcher(statusLine);
boolean hasCode = matcher.find();
Preconditions.checkArgument(hasCode, "Status code not found in `" + statusLine + "`");
this.code = Integer.parseInt(matcher.group(1));
String header;
this.headers = new HashMap<>();
while (!TextUtils.isEmpty(header = reader.readLine())) {
read += header.length() + 1;
String[] keyValue = header.split(":");
String headerName = keyValue[0].trim();
String headerValue = keyValue[1].trim();
headers.put(headerName, Collections.singletonList(headerValue));
}
read++;
this.contentType = headers.containsKey(CONTENT_TYPE_HEADER) ? headers.get(CONTENT_TYPE_HEADER).get(0) : null;
this.contentLength = headers.containsKey(CONTENT_LENGTH_HEADER) ? Integer.parseInt(headers.get(CONTENT_LENGTH_HEADER).get(0)) : -1;
int bodySize = responseData.length - read;
this.data = new byte[bodySize];
System.arraycopy(responseData, read, data, 0, bodySize);
}
}