The CloudDB Component

CloudDB is similar to the FirebaseDB component but does not require a
proprietary back-end database. Instead the back-end is based on the open
source Redis system which is a well used and scale-able data storage
solution.

With CloudDB you can store arbitrary data, including binary image data,
based on a “tag” which is a string. The “tag” is combined with
ProjectID (which defaults to the name of the current project, but can be
changed) so that multiple projects can use the same tags without
interfering with each other.

The domain name system name of the Redis server should be configured the
“RedisServer” property. This defaults to “DEFAULT” which is a server
provided default (more on that below). The “Token” property should be
the password required for the Redis Server. An “out of the box” Redis
server does not implement SSL, so the UseSSL property should be
unchecked.

In the DEFAULT setup, the server name is set to DEFAULT, UseSSL is
checked and the Token is automatically generated when you load a
project. In the DEFAULT case, a special modified Redis server is used
which implements multi-user features which provide isolation between
different MIT App Inventor programmers. It also uses SSL to protect
on-line communications from interception.

Author: Natalie Lao <natalie@csail.mit.edu>
Author: Jeffrey I. Schiller <jis@mit.edu>
Author: Joy Mitra <joymitro1989@gmail.com>

Change-Id: Ic7d7aa0f7337a5452d9c138920813deb797fac20

appinventor/appengine/src/com/google/appinventor/client/Images.java

@@ -538,4 +538,10 @@ public interface Images extends Resources {
   @Source("com/google/appinventor/images/proximitysensor.png")
   ImageResource proximitysensor();
 
+  /**
+   * Designer palette item: cloudDB component
+   */
+  @Source("com/google/appinventor/images/cloudDB.png")
+  ImageResource cloudDB();
+
 }

appinventor/appengine/src/com/google/appinventor/client/Ode.java

@@ -55,6 +55,8 @@ import com.google.appinventor.client.wizards.NewProjectWizard.NewProjectCommand;
 import com.google.appinventor.client.wizards.TemplateUploadWizard;
 import com.google.appinventor.common.version.AppInventorFeatures;
 import com.google.appinventor.components.common.YaVersion;
+import com.google.appinventor.shared.rpc.cloudDB.CloudDBAuthService;
+import com.google.appinventor.shared.rpc.cloudDB.CloudDBAuthServiceAsync;
 import com.google.appinventor.shared.rpc.component.ComponentService;
 import com.google.appinventor.shared.rpc.component.ComponentServiceAsync;
 import com.google.appinventor.shared.rpc.GetMotdService;
@@ -266,6 +268,9 @@ public class Ode implements EntryPoint {
   private final ComponentServiceAsync componentService = GWT.create(ComponentService.class);
   private final AdminInfoServiceAsync adminInfoService = GWT.create(AdminInfoService.class);
 
+  //Web service for CloudDB authentication operations
+  private final CloudDBAuthServiceAsync cloudDBAuthService = GWT.create(CloudDBAuthService.class);
+
   private boolean windowClosing;
 
   private boolean screensLocked;
@@ -1354,6 +1359,15 @@ public class Ode implements EntryPoint {
     return componentService;
   }
 
+  /**
+   * Get an instance of the CloudDBAuth web service.
+   *
+   * @return CloudDBAuth web service instance
+   */
+  public CloudDBAuthServiceAsync getCloudDBAuthService(){
+    return cloudDBAuthService;
+  }
+
   /**
    * Set the current file editor.
    *

appinventor/appengine/src/com/google/appinventor/client/OdeMessages.java

@@ -5382,6 +5382,14 @@ public interface OdeMessages extends Messages {
   @Description("")
   String RemoveFirstMethods();
 
+  @DefaultMessage("AppendValueToList")
+  @Description("")
+  String AppendValueToListMethods();
+
+  @DefaultMessage("RemoveFirstFromList")
+  @Description("")
+  String RemoveFirstFromListMethods();
+
   @DefaultMessage("FirstRemoved")
   @Description("")
   String FirstRemovedEvents();
@@ -6455,6 +6463,63 @@ public interface OdeMessages extends Messages {
   @Description("")
   String reloadWindow();
 
+  @DefaultMessage("AccountName")
+  @Description("")
+  String AccountNameProperties();
+
+  @DefaultMessage("ProjectID")
+  @Description("")
+  String ProjectIDProperties();
+
+  @DefaultMessage("CloudDBError")
+  @Description("")
+  String CloudDBErrorEvents();
+
+  @DefaultMessage("CloudDB")
+  @Description("")
+  String cloudDBComponentPallette();
+
+  @DefaultMessage("Non-visible component allowing you to store data on a Internet " +
+    "connected database server (using Redis software). This allows the users of " +
+    "your App to share data with each other. " +
+    "By default data will be stored in a server maintained by MIT, however you " +
+    "can setup and run your own server. Set the \"RedisServer\" property and " +
+    "\"RedisPort\" Property to access your own server.")
+  @Description("")
+  String CloudDBHelpStringComponentPallette();
+
+  @DefaultMessage("RedisServer")
+  @Description("")
+  String RedisServerProperties();
+
+  @DefaultMessage("DefaultRedisServer")
+  @Description("")
+  String DefaultRedisServerProperties();
+
+  @DefaultMessage("RedisPort")
+  @Description("")
+  String RedisPortProperties();
+
+  @DefaultMessage("Token")
+  @Description("")
+  String TokenProperties();
+
+  @DefaultMessage("GetValues")
+  @Description("")
+  String GetValuesMethods();
+
+  @DefaultMessage("itemToAdd")
+  @Description("")
+  String itemToAddParams();
+
+  @DefaultMessage("UseSSL")
+  @Description("")
+  String UseSSLProperties();
+
+  @DefaultMessage("CloudConnected")
+  @Description("")
+  String CloudConnectedMethods();
+
   @DefaultMessage("PrimaryColor")
   @Description("")
   String PrimaryColorProperties();

appinventor/appengine/src/com/google/appinventor/client/editor/simple/components/MockCloudDB.java

+// -*- mode: java; c-basic-offset: 2; -*-
+// Copyright 2017 MIT, All rights reserved
+// Released under the Apache License, Version 2.0
+// http://www.apache.org/licenses/LICENSE-2.0
+
+package com.google.appinventor.client.editor.simple.components;
+
+import com.google.appinventor.client.DesignToolbar;
+import com.google.appinventor.client.Ode;
+import com.google.appinventor.client.OdeAsyncCallback;
+import com.google.appinventor.client.editor.simple.SimpleEditor;
+
+import com.google.appinventor.client.output.OdeLog;
+
+import com.google.appinventor.client.widgets.properties.EditableProperty;
+
+import com.google.gwt.user.client.ui.Image;
+import com.google.gwt.user.client.ui.Widget;
+
+/**
+ * Mock for the non-visible CloudDB component. This needs a separate mock
+ * from other non-visible components so that some of its properties can be
+ * given dynamic default values.
+ *
+ * @author natalie@csail.mit.edu (Natalie Lao)
+ */
+public class MockCloudDB extends MockNonVisibleComponent {
+
+  public static final String TYPE = "CloudDB";
+  private static final String PROPERTY_NAME_PROJECT_ID = "ProjectID";
+  private static final String PROPERTY_NAME_ACCOUNT_NAME = "AccountName";
+  private static final String PROPERTY_NAME_TOKEN = "Token";
+  private static final String PROPERTY_NAME_REDIS_SERVER = "RedisServer";
+  private static final String PROPERTY_NAME_DEFAULT_REDISSERVER = "DefaultRedisServer";
+
+  private boolean persistToken = false;
+
+  /**
+   * Creates a new instance of a non-visible component whose icon is
+   * loaded dynamically (not part of the icon image bundle)
+   *
+   * @param editor
+   * @param type
+   * @param iconImage
+   */
+  public MockCloudDB(SimpleEditor editor, String type, Image iconImage) {
+    super(editor, type, iconImage);
+  }
+
+  /**
+   * Initializes the "ProjectID", "AccountName" properties dynamically.
+   *
+   * @param widget the iconImage for the MockCloudDB
+   */
+  @Override
+  public final void initComponent(Widget widget) {
+    super.initComponent(widget);
+    String accName = Ode.getInstance().getUser().getUserEmail() + "";
+    DesignToolbar.DesignProject currentProject = Ode.getInstance().getDesignToolbar().getCurrentProject();
+    String projectID = "";
+    if (currentProject != null) {
+      projectID = currentProject.name;
+    }
+
+    changeProperty(PROPERTY_NAME_PROJECT_ID, projectID);
+    changeProperty(PROPERTY_NAME_ACCOUNT_NAME, accName);
+    String defaultRedisServer = Ode.getInstance().getSystemConfig().getDefaultCloudDBserver();
+    changeProperty(PROPERTY_NAME_DEFAULT_REDISSERVER, defaultRedisServer);
+    getTokenFromServer();       // Get Token from the server
+  }
+
+  @Override
+  public boolean isPropertyforYail(String propertyName) {
+    if (propertyName.equals(PROPERTY_NAME_ACCOUNT_NAME) ||
+      (propertyName.equals(PROPERTY_NAME_PROJECT_ID)) ||
+      (propertyName.equals(PROPERTY_NAME_DEFAULT_REDISSERVER)) ||
+      (propertyName.equals(PROPERTY_NAME_TOKEN))) {
+      return true;
+    }
+    return super.isPropertyforYail(propertyName);
+  }
+
+  @Override
+  protected boolean isPropertyVisible(String propertyName) {
+    return !propertyName.equals(PROPERTY_NAME_DEFAULT_REDISSERVER)
+      && super.isPropertyVisible(propertyName);
+  }
+
+  @Override
+  public boolean isPropertyPersisted(String propertyName) {
+    if (propertyName.equals(PROPERTY_NAME_DEFAULT_REDISSERVER)) {
+      return false;             // We don't persist the default server as it is really
+                                // a property of the service, not the project per se
+    } else if (propertyName.equals(PROPERTY_NAME_TOKEN)) {
+      return persistToken;
+    } else {
+      return super.isPropertyPersisted(propertyName);
+    }
+  }
+
+  // We provide our own onPropertyChange to catch the case where the
+  // RedisServer is changed to/from the DEFAULT value.  This effects
+  // the persistence of the Token property. If we are using a private
+  // redis server, we want to persist the Token. Otherwise we do not.
+
+  @Override
+  public void onPropertyChange(String propertyName, String newValue) {
+    if (propertyName.equals(PROPERTY_NAME_REDIS_SERVER)) {
+      // If this is the default server, then make the Token property
+      // non-persistent, but output it in YAIL
+      EditableProperty token = properties.getProperty(PROPERTY_NAME_TOKEN);
+      if (token == null) {      // First pass through and "Token" isn't set yet
+        super.onPropertyChange(propertyName, newValue);
+        return;
+      }
+      int tokenType = token.getType();
+      if (newValue.equals("DEFAULT")) {
+        if (token.getValue().isEmpty() || !(token.getValue().substring(0, 1).equals("%"))) {
+          token.setValue("");   // Set it to empty so getTokenFromServer will fill in
+          persistToken = false;
+          tokenType |= EditableProperty.TYPE_NONPERSISTED;
+          tokenType |= EditableProperty.TYPE_DOYAIL;
+          getTokenFromServer(); // will fill it in.
+        } else {
+          tokenType &= ~EditableProperty.TYPE_NONPERSISTED;
+          persistToken = true;
+        }
+      } else {
+        tokenType &= ~EditableProperty.TYPE_NONPERSISTED;
+        persistToken = true;
+      }
+      token.setType(tokenType);
+      onPropertyChange(PROPERTY_NAME_TOKEN, token.getValue());
+    } else if (propertyName.equals(PROPERTY_NAME_TOKEN)) {
+      EditableProperty serverProperty = properties.getProperty(PROPERTY_NAME_REDIS_SERVER);
+      EditableProperty token = properties.getProperty(PROPERTY_NAME_TOKEN);
+      if (token == null) {      // First pass through and "Token" isn't set yet
+        super.onPropertyChange(propertyName, newValue);
+        return;
+      }
+      int tokenType = token.getType();
+      // if the Redis Server property is "DEFAULT" we don't persist the
+      // Token property
+      if (serverProperty == null) { // Nothing we can do here
+        super.onPropertyChange(propertyName, newValue);
+        return;
+      }
+      String server = serverProperty.getValue();
+      if (server.equals("DEFAULT")) {
+        if (newValue == null || newValue.isEmpty() ||
+          !(newValue.substring(0, 1).equals("%"))) {
+          persistToken = false; // Now that the auto-save is scheduled, we no longer want
+                                // to persist the token
+          tokenType |= EditableProperty.TYPE_NONPERSISTED;
+          tokenType |= EditableProperty.TYPE_DOYAIL;
+        } else {
+          tokenType &= ~EditableProperty.TYPE_NONPERSISTED;
+          persistToken = true;
+        }
+      } else {
+        tokenType &= ~EditableProperty.TYPE_NONPERSISTED;
+        persistToken = true;
+      }
+      token.setType(tokenType);
+    }
+    super.onPropertyChange(propertyName, newValue);
+  }
+
+  private void getTokenFromServer() {
+    Ode.getInstance().getCloudDBAuthService().getToken(new OdeAsyncCallback<String>() {
+      @Override
+      public void onSuccess(String token) {
+        EditableProperty tokenProperty = MockCloudDB.this.properties.getProperty(PROPERTY_NAME_TOKEN);
+        if (tokenProperty != null) {
+          String existingToken = tokenProperty.getValue();
+          if (!existingToken.isEmpty()) {
+            return;             // If we have a value, don't over-write it
+          }
+        }
+        changeProperty(PROPERTY_NAME_TOKEN, token);
+      }
+      @Override
+      public void onFailure(Throwable t){
+        changeProperty(PROPERTY_NAME_TOKEN, "ERROR : token not created");
+        super.onFailure(t);
+      }
+    });
+  }
+
+}

appinventor/appengine/src/com/google/appinventor/client/editor/simple/palette/SimpleComponentDescriptor.java

@@ -14,6 +14,7 @@ import com.google.appinventor.client.editor.simple.components.MockBall;
 import com.google.appinventor.client.editor.simple.components.MockButton;
 import com.google.appinventor.client.editor.simple.components.MockCanvas;
 import com.google.appinventor.client.editor.simple.components.MockCheckBox;
+import com.google.appinventor.client.editor.simple.components.MockCloudDB;
 import com.google.appinventor.client.editor.simple.components.MockComponent;
 import com.google.appinventor.client.editor.simple.components.MockContactPicker;
 import com.google.appinventor.client.editor.simple.components.MockDatePicker;
@@ -42,6 +43,7 @@ import com.google.appinventor.client.editor.simple.components.MockVideoPlayer;
 import com.google.appinventor.client.editor.simple.components.MockWebViewer;
 
 import com.google.appinventor.shared.storage.StorageUtil;
+
 import com.google.common.collect.Maps;
 
 import com.google.gwt.resources.client.ImageResource;
@@ -142,6 +144,7 @@ public final class SimpleComponentDescriptor {
     bundledImages.put("images/yandex.png", images.yandex());
     bundledImages.put("images/proximitysensor.png", images.proximitysensor());
     bundledImages.put("images/extension.png", images.extension());
+    bundledImages.put("images/cloudDB.png", images.cloudDB());
 
     imagesInitialized = true;
   }
@@ -327,12 +330,16 @@ public final class SimpleComponentDescriptor {
       if(name.equals(MockFirebaseDB.TYPE)) {
         return new MockFirebaseDB(editor, name,
           getImageFromPath(SimpleComponentDatabase.getInstance(editor.getProjectId()).getIconName(name),
-              null, editor.getProjectId()));
+            null, editor.getProjectId()));
+      } else if(name.equals(MockCloudDB.TYPE)) {
+        return new MockCloudDB(editor, name,
+          getImageFromPath(SimpleComponentDatabase.getInstance(editor.getProjectId()).getIconName(name),
+            null, editor.getProjectId()));
       } else {
         String pkgName = type.contains(".") ? type.substring(0, type.lastIndexOf('.')) : null;
         return new MockNonVisibleComponent(editor, name,
           getImageFromPath(SimpleComponentDatabase.getInstance(editor.getProjectId()).getIconName(name),
-              pkgName, editor.getProjectId()));
+            pkgName, editor.getProjectId()));
       }
     } else if (name.equals(MockButton.TYPE)) {
       return new MockButton(editor);

appinventor/appengine/src/com/google/appinventor/server/OdeRemoteServiceServlet.java

@@ -9,7 +9,6 @@ package com.google.appinventor.server;
 import com.google.appinventor.shared.rpc.user.UserInfoProvider;
 import com.google.gwt.user.server.rpc.RemoteServiceServlet;
 
-
 /**
  * Class supporting ODE GWT RPC, which ODE RPC
  * implementations should subclass instead of

appinventor/appengine/src/com/google/appinventor/server/UserInfoServiceImpl.java

@@ -61,6 +61,7 @@ public class UserInfoServiceImpl extends OdeRemoteServiceServlet implements User
     config.setGuideUrl(Flag.createFlag("guide.url", "").get());
     config.setReferenceComponentsUrl(Flag.createFlag("reference.components.url", "").get());
     config.setFirebaseURL(Flag.createFlag("firebase.url", "").get());
+    config.setDefaultCloudDBserver(Flag.createFlag("clouddb.server", "").get());
     config.setNoop(Flag.createFlag("session.noop", 0).get());
 
     // Check to see if we need to upgrade this user's project to GCS
@@ -153,7 +154,7 @@ public class UserInfoServiceImpl extends OdeRemoteServiceServlet implements User
 
   /**
    * Stores the user's link.
-   * @param name  user's link
+   * @param link  user's link
    */
   @Override
   public void storeUserLink(String link) {
@@ -193,5 +194,4 @@ public class UserInfoServiceImpl extends OdeRemoteServiceServlet implements User
   @Override
   public void noop() {
   }
-
 }

appinventor/appengine/src/com/google/appinventor/server/cloudDBAuth/CloudDBAuthServiceImpl.java

+// -*- mode: java; c-basic-offset: 2; -*-
+// Copyright 2017 MIT, All rights reserved
+// Released under the Apache License, Version 2.0
+// http://www.apache.org/licenses/LICENSE-2.0
+
+package com.google.appinventor.server.cloudDBAuth;
+
+import com.google.appinventor.server.OdeRemoteServiceServlet;
+import com.google.appinventor.server.flags.Flag;
+
+import com.google.appinventor.shared.rpc.cloudDB.CloudDBAuthService;
+
+import com.google.appinventor.shared.util.Base58Util;
+
+import com.google.protobuf.ByteString;
+
+import java.security.InvalidKeyException;
+import java.security.NoSuchAlgorithmException;
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+
+/**
+ * CloudDB Authentication Service implementation
+ * @author joymitro1989@gmail.com(Joydeep Mitra).
+ */
+public class CloudDBAuthServiceImpl extends OdeRemoteServiceServlet
+  implements CloudDBAuthService {
+
+  private String SECRET_KEY_UUID = Flag.createFlag("clouddb.uuid.secret", "").get();
+  private String SECRET_KEY_CLOUD_DB = Flag.createFlag("clouddb.secret", "").get();
+  private static final String HMAC_ALGORITHM = "HmacSHA256";
+
+  /*
+   * returns the auth token for CloudDB encoded in base58.
+   */
+  @Override
+  public String getToken() {
+    byte [] hunsigned = createUnsigned(getHuuid()).toByteArray();
+    if (hunsigned != null) {
+      try {
+        SecretKeySpec secretKeySpec = new SecretKeySpec(SECRET_KEY_CLOUD_DB.getBytes(), HMAC_ALGORITHM);
+        Mac hmac = Mac.getInstance(HMAC_ALGORITHM);
+        hmac.init(secretKeySpec);
+        TokenAuth.token token = createToken(hunsigned,hmac.doFinal(hunsigned));
+        return Base58Util.encode(token.toByteArray());
+      } catch (NoSuchAlgorithmException e) {
+        e.printStackTrace();
+        return null;
+      } catch (InvalidKeyException e) {
+        e.printStackTrace();
+        return null;
+      } catch(Exception e){
+        e.printStackTrace();
+        return null;
+      }
+    }
+    return null;
+  }
+
+  /*
+    returns hashed userId
+  */
+  private String getHuuid(){
+    try {
+      SecretKeySpec secretKeySpec = new SecretKeySpec(SECRET_KEY_UUID.getBytes(), HMAC_ALGORITHM);
+      Mac hmac = Mac.getInstance(HMAC_ALGORITHM);
+      hmac.init(secretKeySpec);
+      return Base58Util.encode(hmac.doFinal(userInfoProvider.getUserId().getBytes()));
+    } catch (NoSuchAlgorithmException e) {
+      e.printStackTrace();
+      return null;
+    } catch(InvalidKeyException e) {
+      e.printStackTrace();
+      return null;
+    }
+  }
+
+  /*
+    returns a Token as a Google Protocol Buffer object
+  */
+  private TokenAuth.token createToken(byte[] unsigned, byte[] signature){
+    TokenAuth.token token = TokenAuth.token.newBuilder().setVersion(1)
+      .setKeyid(1)
+      .setUnsigned(ByteString.copyFrom(unsigned))
+      .setSignature(ByteString.copyFrom(signature)).build();
+    return token;
+  }
+
+  private TokenAuth.unsigned createUnsigned(String huuid) {
+    TokenAuth.unsigned retval = TokenAuth.unsigned.newBuilder().setHuuid(huuid).build();
+    return retval;
+  }
+}

appinventor/appengine/src/com/google/appinventor/server/cloudDBAuth/token.proto

+package tokenauth;
+
+option java_package = "com.google.appinventor.server.cloudDBAuth";
+option java_outer_classname = "TokenAuth";
+
+// This message just contains the user's uuid, which is
+// unique to them. Depending on App Inventor version this
+// will either look like a large integer (we use its decimal
+// representation) or a uuid.
+
+message unsigned {
+    optional string huuid = 1;
+}
+
+// This is the actual message token. The "unsigned" field
+// contains the serialized version of the "unsigned" message
+// above. The "signature" fields contains the raw bytes of
+// the output of HMAC-SHA1 using the key identified by
+// "keyid"
+
+// In actual use this token is serialized and then base58
+// encoded.
+
+message token {
+    required uint64 version = 1 [default = 1];
+    optional uint64 keyid = 2;
+    optional bytes unsigned = 3;
+    optional bytes signature = 4;
+}
\ No newline at end of file

appinventor/appengine/src/com/google/appinventor/shared/rpc/ServerLayout.java

@@ -242,6 +242,13 @@ public class ServerLayout {
    */
   public static final String UPLOAD_USERFILE_FORM_ELEMENT = "uploadUserFile";
 
+  /**
+   * Relative path of the
+   * {@link com.google.appinventor.shared.rpc.cloudDB.CloudDBAuthService} within the
+   * ODE GWT module.
+   */
+  public static final String CLOUD_DB_AUTH_SERVICE = "cloudDBAuth";
+
   public static String genRelativeDownloadPath(long projectId, String target) {
     return DOWNLOAD_SERVLET_BASE + DOWNLOAD_PROJECT_OUTPUT + "/" + projectId + "/" + target;
   }

appinventor/appengine/src/com/google/appinventor/shared/rpc/cloudDB/CloudDBAuthService.java

+package com.google.appinventor.shared.rpc.cloudDB;
+
+import com.google.appinventor.shared.rpc.ServerLayout;
+import com.google.gwt.user.client.rpc.RemoteService;
+import com.google.gwt.user.client.rpc.RemoteServiceRelativePath;
+
+/**
+ *  Service interface for the CloudDB authentication RPC.
+ * @author joymitro1989@gmail.com (Joydeep Mitra).
+ */
+@RemoteServiceRelativePath(ServerLayout.CLOUD_DB_AUTH_SERVICE)
+public interface CloudDBAuthService extends RemoteService{
+
+    String getToken();
+}

appinventor/appengine/src/com/google/appinventor/shared/rpc/cloudDB/CloudDBAuthServiceAsync.java

+package com.google.appinventor.shared.rpc.cloudDB;
+
+import com.google.gwt.user.client.rpc.AsyncCallback;
+
+/**
+ * Interface for the service providing user related information. All
+ * declarations in this interface are mirrored in {@link CloudDBAuthService}.
+ * For further information see {@link CloudDBAuthService}.
+ *
+ * @author joymitro1989@gmail.com (Joydeep Mitra)
+ */
+public interface CloudDBAuthServiceAsync {
+
+    void getToken(AsyncCallback<String> callback);
+}

appinventor/appengine/src/com/google/appinventor/shared/rpc/user/Config.java

@@ -33,6 +33,7 @@ public class Config implements IsSerializable, Serializable {
   private String guideUrl;
   private String referenceComponentsUrl;
   private String firebaseURL;   // Default Firebase URL
+  private String defaultCloudDBserver;
   private int noop;            // No-op interval
 
   public Config() {
@@ -166,6 +167,14 @@ public class Config implements IsSerializable, Serializable {
     firebaseURL = url;
   }
 
+  public void setDefaultCloudDBserver(String server) {
+    defaultCloudDBserver = server;
+  }
+
+  public String getDefaultCloudDBserver() {
+    return defaultCloudDBserver;
+  }
+
   public int getNoop() {
     return noop;
   }

appinventor/appengine/src/com/google/appinventor/shared/util/AddressFormatException.java

+package com.google.appinventor.shared.util;
+
+/*
+ * Copyright 2011 Google Inc.
+ * Copyright 2015 Andreas Schildbach
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+public class AddressFormatException extends IllegalArgumentException {
+
+    @SuppressWarnings("serial")
+    public AddressFormatException() {
+        super();
+    }
+
+    public AddressFormatException(String message) {
+        super(message);
+    }
+}

appinventor/appengine/src/com/google/appinventor/shared/util/Base58Util.java

+/*
+ * Copyright 2011 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.google.appinventor.shared.util;
+import java.math.BigInteger;
+import java.util.Arrays;
+
+/**
+ * Base58 is a way to encode Bitcoin addresses (or arbitrary data) as alphanumeric strings.
+ * <p>
+ * Note that this is not the same base58 as used by Flickr, which you may find referenced around the Internet.
+ * <p>
+ * You may want to consider working with {@link VersionedChecksummedBytes} instead, which
+ * adds support for testing the prefix and suffix bytes commonly found in addresses.
+ * <p>
+ * Satoshi explains: why base-58 instead of standard base-64 encoding?
+ * <ul>
+ * <li>Don't want 0OIl characters that look the same in some fonts and
+ *     could be used to create visually identical looking account numbers.</li>
+ * <li>A string with non-alphanumeric characters is not as easily accepted as an account number.</li>
+ * <li>E-mail usually won't line-break if there's no punctuation to break at.</li>
+ * <li>Doubleclicking selects the whole number as one word if it's all alphanumeric.</li>
+ * </ul>
+ * <p>
+ * However, note that the encoding/decoding runs in O(n&sup2;) time, so it is not useful for large data.
+ * <p>
+ * The basic idea of the encoding is to treat the data bytes as a large number represented using
+ * base-256 digits, convert the number to be represented using base-58 digits, preserve the exact
+ * number of leading zeros (which are otherwise lost during the mathematical operations on the
+ * numbers), and finally represent the resulting base-58 digits as alphanumeric ASCII characters.
+ */
+public class Base58Util {
+    public static final char[] ALPHABET = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz".toCharArray();
+    private static final char ENCODED_ZERO = ALPHABET[0];
+    private static final int[] INDEXES = new int[128];
+    static {
+        Arrays.fill(INDEXES, -1);
+        for (int i = 0; i < ALPHABET.length; i++) {
+            INDEXES[ALPHABET[i]] = i;
+        }
+    }
+
+    /**
+     * Encodes the given bytes as a base58 string (no checksum is appended).
+     *
+     * @param input the bytes to encode
+     * @return the base58-encoded string
+     */
+    public static String encode(byte[] input) {
+        if (input.length == 0) {
+            return "";
+        }
+        // Count leading zeros.
+        int zeros = 0;
+        while (zeros < input.length && input[zeros] == 0) {
+            ++zeros;
+        }
+        // Convert base-256 digits to base-58 digits (plus conversion to ASCII characters)
+        input = Arrays.copyOf(input, input.length); // since we modify it in-place
+        char[] encoded = new char[input.length * 2]; // upper bound
+        int outputStart = encoded.length;
+        for (int inputStart = zeros; inputStart < input.length; ) {
+            encoded[--outputStart] = ALPHABET[divmod(input, inputStart, 256, 58)];
+            if (input[inputStart] == 0) {
+                ++inputStart; // optimization - skip leading zeros
+            }
+        }
+        // Preserve exactly as many leading encoded zeros in output as there were leading zeros in input.
+        while (outputStart < encoded.length && encoded[outputStart] == ENCODED_ZERO) {
+            ++outputStart;
+        }
+        while (--zeros >= 0) {
+            encoded[--outputStart] = ENCODED_ZERO;
+        }
+        // Return encoded string (including encoded leading zeros).
+        return new String(encoded, outputStart, encoded.length - outputStart);
+    }
+
+    /**
+     * Decodes the given base58 string into the original data bytes.
+     *
+     * @param input the base58-encoded string to decode
+     * @return the decoded data bytes
+     * @throws AddressFormatException if the given string is not a valid base58 string
+     */
+    public static byte[] decode(String input) throws AddressFormatException {
+        if (input.length() == 0) {
+            return new byte[0];
+        }
+        // Convert the base58-encoded ASCII chars to a base58 byte sequence (base58 digits).
+        byte[] input58 = new byte[input.length()];
+        for (int i = 0; i < input.length(); ++i) {
+            char c = input.charAt(i);
+            int digit = c < 128 ? INDEXES[c] : -1;
+            if (digit < 0) {
+                throw new AddressFormatException("Illegal character " + c + " at position " + i);
+            }
+            input58[i] = (byte) digit;
+        }
+        // Count leading zeros.
+        int zeros = 0;
+        while (zeros < input58.length && input58[zeros] == 0) {
+            ++zeros;
+        }
+        // Convert base-58 digits to base-256 digits.
+        byte[] decoded = new byte[input.length()];
+        int outputStart = decoded.length;
+        for (int inputStart = zeros; inputStart < input58.length; ) {
+            decoded[--outputStart] = divmod(input58, inputStart, 58, 256);
+            if (input58[inputStart] == 0) {
+                ++inputStart; // optimization - skip leading zeros
+            }
+        }
+        // Ignore extra leading zeroes that were added during the calculation.
+        while (outputStart < decoded.length && decoded[outputStart] == 0) {
+            ++outputStart;
+        }
+        // Return decoded data (including original number of leading zeros).
+        return Arrays.copyOfRange(decoded, outputStart - zeros, decoded.length);
+    }
+
+    public static BigInteger decodeToBigInteger(String input) throws AddressFormatException {
+        return new BigInteger(1, decode(input));
+    }
+
+
+
+    /**
+     * Divides a number, represented as an array of bytes each containing a single digit
+     * in the specified base, by the given divisor. The given number is modified in-place
+     * to contain the quotient, and the return value is the remainder.
+     *
+     * @param number the number to divide
+     * @param firstDigit the index within the array of the first non-zero digit
+     *        (this is used for optimization by skipping the leading zeros)
+     * @param base the base in which the number's digits are represented (up to 256)
+     * @param divisor the number to divide by (up to 256)
+     * @return the remainder of the division operation
+     */
+    private static byte divmod(byte[] number, int firstDigit, int base, int divisor) {
+        // this is just long division which accounts for the base of the input digits
+        int remainder = 0;
+        for (int i = firstDigit; i < number.length; i++) {
+            int digit = (int) number[i] & 0xFF;
+            int temp = remainder * base + digit;
+            number[i] = (byte) (temp / divisor);
+            remainder = temp % divisor;
+        }
+        return (byte) remainder;
+    }
+
+}
\ No newline at end of file

appinventor/appengine/war/WEB-INF/appengine-web.xml

@@ -163,6 +163,11 @@
     <property name="guide.url" value="http://appinventor.mit.edu/explore/library" />
     <property name="reference.components.url" value="/reference/components/" />
 
+    <!-- CloudDB secret keys -->
+    <property name="clouddb.server" value="clouddb.example.com"/>
+    <property name="clouddb.uuid.secret" value="changeme!"/>
+    <property name="clouddb.secret" value="changeme too!" />
+
   </system-properties>
 
   <!-- Enable concurrency in the app engine server -->

appinventor/appengine/war/WEB-INF/web.xml

@@ -251,6 +251,20 @@
     <servlet-name>userInfoService</servlet-name>
   </filter-mapping>
 
+  <!-- cloudDBAuth -->
+  <servlet>
+    <servlet-name>cloudDBAuthService</servlet-name>
+    <servlet-class>com.google.appinventor.server.cloudDBAuth.CloudDBAuthServiceImpl</servlet-class>
+  </servlet>
+  <servlet-mapping>
+    <servlet-name>cloudDBAuthService</servlet-name>
+    <url-pattern>/ode/cloudDBAuth</url-pattern>
+  </servlet-mapping>
+  <filter-mapping>
+    <filter-name>odeAuthFilter</filter-name>
+    <servlet-name>cloudDBAuthService</servlet-name>
+  </filter-mapping>
+
   <!-- android -->
   <servlet>
     <servlet-name>androidServlet</servlet-name>

appinventor/blocklyeditor/src/versioning.js

@@ -1358,6 +1358,13 @@ Blockly.Versioning.AllUpgradeMaps =
 
   }, // End Clock upgraders
 
+  "CloudDB": {
+
+    //This is initial version. Placeholder for future upgrades
+    1: "noUpgrade"
+
+  },
+
   "ContactPicker": {
 
     // AI1: The Alignment property was renamed to TextAlignment.

appinventor/buildserver/build.xml

@@ -82,6 +82,8 @@
     <copy toFile="${classes.files.dir}/gson-2.1.jar" file="${lib.dir}/gson/gson-2.1.jar" />
     <copy toFile="${classes.files.dir}/json.jar" file="${lib.dir}/json/json.jar" />
     <copy toFile="${classes.files.dir}/google-oauth-client-beta.jar" file="${lib.dir}/oauth/google-oauth-client-1.10.1-beta.jar" />
+    <copy toFile="${classes.files.dir}/jedis.jar" file="${lib.dir}/jedis/jedis-3.0.0-SNAPSHOT-jar-with-dependencies.jar" />
+    <copy toFile="${classes.files.dir}/commons-pool.jar" file="${lib.dir}/commons-pool/commons-pool2-2.0.jar" />
     <copy toFile="${classes.files.dir}/guava-14.0.1.jar" file="${lib.dir}/guava/guava-14.0.1.jar" />
     <copy toFile="${classes.files.dir}/core.jar" file="${lib.dir}/QRGenerator/core.jar" />
     <copy toFile="${classes.files.dir}/appcompat-v7.aar" file="${lib.dir}/android/support/appcompat-v7-22.2.1.aar" />

appinventor/components/build.xml

@@ -154,6 +154,8 @@
         <pathelement location="${lib.dir}/oauth/google-http-client-android2-1.10.3-beta.jar" />
         <pathelement location="${lib.dir}/oauth/google-http-client-android3-1.10.3-beta.jar" />
         <pathelement location="${lib.dir}/oauth/google-oauth-client-1.10.1-beta.jar" />
+        <pathelement location="${lib.dir}/jedis/jedis-3.0.0-SNAPSHOT-jar-with-dependencies.jar" />
+        <pathelement location="${lib.dir}/commons-pool/commons-pool2-2.0.jar" />
         <pathelement location="${lib.dir}/gson/gson-2.1.jar" />
         <pathelement location="${lib.dir}/json/json.jar" />
       </classpath>
@@ -271,6 +273,8 @@
           <pathelement location="${lib.dir}/oauth/google-http-client-android2-1.10.3-beta.jar" />
           <pathelement location="${lib.dir}/oauth/google-http-client-android3-1.10.3-beta.jar" />
           <pathelement location="${lib.dir}/oauth/google-oauth-client-1.10.1-beta.jar" />
+          <pathelement location="${lib.dir}/jedis/jedis-3.0.0-SNAPSHOT-jar-with-dependencies.jar" />
+          <pathelement location="${lib.dir}/commons-pool/commons-pool2-2.0.jar" />
           <pathelement location="${lib.dir}/guava/guava-14.0.1.jar" />
         </classpath>
         <compilerarg line="-processorpath ${local.build.dir}/AnnotationProcessors.jar"/>

appinventor/components/src/com/google/appinventor/components/common/YaVersion.java

@@ -403,8 +403,10 @@ public class YaVersion {
   // - BLOCKS_LANGUAGE_VERSION was incremented to 21
   // For YOUNG_ANDROID_VERSION 162:
   // - ACCELEROMETERSENSOR_COMPONENT_VERSION was incremented to 4
+  // For YOUNG_ANDROID_VERSION 163:
+  // Added CloudDB
 
-  public static final int YOUNG_ANDROID_VERSION = 162;
+  public static final int YOUNG_ANDROID_VERSION = 163;
 
   // ............................... Blocks Language Version Number ...............................
 
@@ -981,6 +983,10 @@ public class YaVersion {
   // - Added the ClearTag function, GetTagList and Persist
   public static final int FIREBASE_COMPONENT_VERSION = 3;
 
+  // For CLOUDDB_COMPONENT_VERSION 1:
+  // - CloudDB component introduced
+  public static final int CLOUDDB_COMPONENT_VERSION = 1;
+
   // For TWITTER_COMPONENT_VERSION 2:
   // - The Authorize method and IsAuthorized event handler were added to support
   //   OAuth authentication (now requred by Twitter). These

appinventor/components/src/com/google/appinventor/components/runtime/File.java

@@ -254,7 +254,7 @@ public class File extends AndroidNonvisibleComponent implements Component {
    * be nice - in case someone really wants to detect Windows-style
    * line separators, or save a file which was read (and expect no
    * changes in size or checksum).
-   * @param string to convert
+   * @param s to convert
    */
 
   private String normalizeNewLines(String s) {
@@ -265,7 +265,8 @@ public class File extends AndroidNonvisibleComponent implements Component {
   /**
    * Asynchronously reads from the given file. Calls the main event thread
    * when the function has completed reading from the file.
-   * @param filepath the file to read
+   * @param fileInput the stream to read from
+   * @param fileName the file to read
    * @throws FileNotFoundException
    * @throws IOException when the system cannot read the file
    */

appinventor/components/src/com/google/appinventor/components/runtime/Form.java

@@ -186,6 +186,7 @@ public class Form extends AppCompatActivity
   // Application lifecycle related fields
   private final HashMap<Integer, ActivityResultListener> activityResultMap = Maps.newHashMap();
   private final Set<OnStopListener> onStopListeners = Sets.newHashSet();
+  private final Set<OnClearListener> onClearListeners = Sets.newHashSet();
   private final Set<OnNewIntentListener> onNewIntentListeners = Sets.newHashSet();
   private final Set<OnResumeListener> onResumeListeners = Sets.newHashSet();
   private final Set<OnPauseListener> onPauseListeners = Sets.newHashSet();
@@ -667,6 +668,10 @@ public class Form extends AppCompatActivity
     onStopListeners.add(component);
   }
 
+  public void registerForOnClear(OnClearListener component) {
+    onClearListeners.add(component);
+  }
+
   @Override
   protected void onDestroy() {
     super.onDestroy();
@@ -2033,6 +2038,7 @@ public class Form extends AppCompatActivity
 
   // This is called from clear-current-form in runtime.scm.
   public void clear() {
+    Log.d(LOG_TAG, "Form " + formName + " clear called");
     viewLayout.getLayoutManager().removeAllViews();
     if (frameLayout != null) {
       frameLayout.removeAllViews();
@@ -2049,6 +2055,12 @@ public class Form extends AppCompatActivity
     onCreateOptionsMenuListeners.clear();
     onOptionsItemSelectedListeners.clear();
     screenInitialized = false;
+    // Notifiy those who care
+    for (OnClearListener onClearListener : onClearListeners) {
+      onClearListener.onClear();
+    }
+    // And reset the list
+    onClearListeners.clear();
     System.err.println("Form.clear() About to do moby GC!");
     System.gc();
     dimChanges.clear();

appinventor/components/src/com/google/appinventor/components/runtime/OnClearListener.java

+// -*- mode: java; c-basic-offset: 2; -*-
+// Copyright 2017 MIT, All rights reserved
+// Released under the Apache License, Version 2.0
+// http://www.apache.org/licenses/LICENSE-2.0
+
+package com.google.appinventor.components.runtime;
+
+/**
+ * Listener for distributing the Activity onStop() method to interested components.
+ * Listener for components that want to be notified when (clear-current-form) is called
+ * This is only used in the Companion.
+ *
+ * @author jis@mit.edu (Jeffrey I. Schiller)
+ */
+
+public interface OnClearListener {
+  public void onClear();
+}

appinventor/components/src/com/google/appinventor/components/runtime/util/CloudDBJedisListener.java

+// -*- mode: java; c-basic-offset: 2; -*-
+// Copyright 2017 MIT, All rights reserved
+// Released under the Apache License, Version 2.0
+// http://www.apache.org/licenses/LICENSE-2.0
+//Notification Listener from: http://stackoverflow.com/questions/26406303/redis-key-expire-notification-with-jedis
+
+package com.google.appinventor.components.runtime.util;
+
+import android.util.Log;
+
+import com.google.appinventor.components.runtime.CloudDB;
+import com.google.appinventor.components.runtime.util.JsonUtil;
+
+import java.util.List;
+import java.util.Set;
+
+import org.json.JSONException;
+
+import redis.clients.jedis.Jedis;
+import redis.clients.jedis.JedisPubSub;
+
+public class CloudDBJedisListener extends JedisPubSub {
+  private static final boolean DEBUG = false;
+  public CloudDB cloudDB;
+  private Thread myThread;
+  private static String LOG_TAG = "CloudDB"; // Yep, same as the CloudDB component.
+                                             // This is on purpose because when we
+                                             // are looking at logs for CloudDB
+                                             // we want to know about us as well
+
+  public CloudDBJedisListener(CloudDB thisCloudDB){
+    cloudDB = thisCloudDB;
+    myThread = Thread.currentThread();
+  }
+
+  @Override
+  public void onSubscribe(String channel, int subscribedChannels) {
+    if (DEBUG) {
+      Log.d(LOG_TAG, "onSubscribe " + channel + " " + subscribedChannels);
+    }
+  }
+
+  @Override
+  public void onMessage(String channel, String message) {
+    if (DEBUG) {
+      Log.d(LOG_TAG, "onMessage channel " + channel + ", message: " + message);
+    }
+    try {
+    // Message is a JSON encoded list of the tag that was just set and its value
+      List<Object> data = null;
+      data = (List<Object>) JsonUtil.getObjectFromJson((String) message);
+      if (DEBUG) {
+        Log.d(LOG_TAG, "onMessage: data = " + data);
+      }
+      String tag = (String) data.get(0);   // The variable that was changed
+      List<Object> valueList = (List<Object>) data.get(1);
+      for (Object value : valueList) {
+        // Note: DataChanged will arrange to dispatch the event
+        // on the UI thread.
+        cloudDB.DataChanged(tag, value);
+      }
+    } catch (JSONException e) {
+      Log.e(LOG_TAG, "onMessage: JSONException", e);
+      // CloudDBError arranges to generate the error UI on the
+      // UI Thread
+      cloudDB.CloudDBError("System Error: " + e.getMessage());
+    }
+  }
+
+  public void terminate() {
+    myThread.interrupt();
+  }
+
+  //add other Unimplemented methods
+}

appinventor/docs/reference/components/experimental.html

@@ -455,9 +455,61 @@
 <h1>Experimental Components - App Inventor for Android</h1>
 <h2>Table of Contents</h2>
 <ul>
+  <li> <a href="#CloudDB">CloudDB</a</li>
   <li> <a href="#FirebaseDB">FirebaseDB</a></li>
 </ul>
+<h2 id="CloudDB">CloudDB</h2>
 
+<p>Non-visible component allowing you to store data on a Internet connected database server (using Redis software). This allows the users of your App to share data with each other. By default data will be stored in a server maintained by MIT, however you can setup and run your own server. Set the "RedisServer" property and "RedisPort" Property to access your own server.</p>
+
+<h3>Properties</h3>
+<dl>
+  <dt><code><em>ProjectID</em></code></dt>
+  <dd>Gets the ProjectID for this CloudDB project.</dd>
+  <dt><code><em>RedisPort</em></code></dt>
+  <dd>The Redis Server port to use. Defaults to 6381</dd>
+  <dt><code><em>RedisServer</em></code></dt>
+  <dd>The Redis Server to use to store data. A setting of "DEFAULT" means that the MIT server will be used.</dd>
+  <dt><code>Token</code> (designer only)</dt>
+  <dd>This field contains the authentication token used to login to the backed Redis server. For the "DEFAULT" server, do not edit this value, the system will fill it in for you. A system administrator may also provide a special value to you which can be used to share data between multiple projects from multiple people. If using your own Redis server, set a password in the server's config and enter it here.</dd>
+  <dt><code>UseSSL</code> (designer only)</dt>
+  <dd>Set to true to use SSL to talk to CloudDB/Redis server. This should be set to True for the "DEFAULT" server.</dd>
+</dl>
+
+<h3>Events</h3>
+<dl>
+  <dt><code>CloudDBError(text message)</code></dt>
+  <dd>Indicates that an error occurred while communicating with the CloudDB Redis server.</dd>
+  <dt><code>DataChanged(text tag, any value)</code></dt>
+  <dd>Indicates that the data in the CloudDB project has changed.
+ Launches an event with the tag and value that have been updated.</dd>
+  <dt><code>FirstRemoved(any value)</code></dt>
+  <dd>Event triggered by the "RemoveFirstFromList" function. The argument "value" is the object that was the first in the list, and which is now removed.</dd>
+  <dt><code>GotValue(text tag, any value)</code></dt>
+  <dd>Indicates that a GetValue request has succeeded.</dd>
+  <dt><code>TagList(list value)</code></dt>
+  <dd>Event triggered when we have received the list of known tags. Used with the "GetTagList" Function.</dd>
+</dl>
+
+<h3>Methods</h3>
+<dl>
+  <dt><code>AppendValueToList(text tag, any itemToAdd)</code></dt>
+  <dd>Append a value to the end of a list atomically. If two devices use this function simultaneously, both will be appended and no data lost.</dd>
+  <dt><code>ClearTag(text tag)</code></dt>
+  <dd>Remove the tag from CloudDB</dd>
+  <dt><code>boolean CloudConnected()</code></dt>
+  <dd>returns True if we are on the network and will likely be able to connect to the CloudDB server.</dd>
+  <dt><code>GetTagList()</code></dt>
+  <dd>Get the list of tags for this application. When complete a "TagList" event will be triggered with the list of known tags.</dd>
+  <dt><code>GetValue(text tag, any valueIfTagNotThere)</code></dt>
+  <dd>GetValue asks CloudDB to get the value stored under the given tag.
+ It will pass valueIfTagNotThere to GotValue if there is no value stored
+ under the tag.</dd>
+  <dt><code>RemoveFirstFromList(text tag)</code></dt>
+  <dd>Return the first element of a list and atomically remove it. If two devices use this function simultaneously, one will get the first element and the the other will get the second element, or an error if there is no available element. When the element is available, the "FirstRemoved" event will be triggered.</dd>
+  <dt><code>StoreValue(text tag, any valueToStore)</code></dt>
+  <dd>Asks CloudDB to store the given value under the given tag.</dd>
+</dl>
 <h2 id="FirebaseDB">FirebaseDB</h2>
 
 <p>Non-visible component that communicates with Firebase to store and retrieve information.</p>