Efficient Model Training in the Cloud with Kubernetes, TensorFlow, and Alluxio

Alibaba Cloud Container Service Team Case Study

This article presents the collaboration of Alibaba, Alluxio, and Nanjing University in tackling the problem of Deep Learning model training in the cloud. Various performance bottlenecks are analyzed with detailed optimizations of each component in the architecture. Our goal was to reduce the cost and complexity of data access for Deep Learning training in a hybrid environment, which resulted in an over 40% reduction in training time and cost.

1. New Trends in AI: Kubernetes-Based Deep Learning in The Cloud

Background

Artificial neural networks are trained with increasingly massive amounts of data, driving innovative solutions to improve data processing. Distributed Deep Learning (DL) model training can take advantage of multiple technologies, such as:

Extension Functions and Properties in Kotlin

Introduction

In Kotlin, we can extend classes without subclassing via extension functions and extension properties. This feature is useful whenever we want to extend 3rd party libraries or final classes.

In this tutorial, we will learn what extension functions and extension properties are, and how to create our own.

Goals

At the end of the tutorial, you would have learned:

  1. What extension functions are.
  2. What extension properties are.
  3. How to create extension functions.
  4. How to create extension properties.
Prerequisite Knowledge
  1. Basic Kotlin.
Tools Required
  1. An IDE that supports Kotlin such as IntelliJ Community Edition.
Project Setup

To follow along with this tutorial, perform the steps below:

  1. Create a new Kotlin project using Gradle as the build system.

  2. For the Project JDK, use JDK 16.

  3. After your Gradle project is loaded, copy and paste the content for the build.gradle.kts file below.

     import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
    
     plugins {
        kotlin("jvm") version "1.5.31"
     }
    
     group = "com.example"
     version = "1.0-SNAPSHOT"
    
     repositories {
        mavenCentral()
     }
    
     dependencies {
        testImplementation(kotlin("test"))
     }
    
     tasks.test {
        useJUnitPlatform()
     }
    
     tasks.withType<KotlinCompile>() {
        kotlinOptions.jvmTarget = "11"
     }
  4. Under src/main/kotlin, create a new package called com.example.

  5. Under com.example, create an Entry.kt file.

  6. Create the main() function with zero argument in this file.

Extension Functions Concept Overview

To understand how extension functions work, let us start with a scenario where extending a class is not allowed. In the Entry.kt file, below the main() function, create a class called PizzaMaker using the code below.

    class PizzaMaker {

       fun makePepperoniPizza(){
           println("Pepperoni Pizza")
       }

       fun makeHawaiianPizza(){
           println("Hawaiian Pizza")
       }
    }

The class above only contains two simple functions, which are to make Pepperoni and Hawaiian pizzas.

If you are coming from a Java world, you might have thought that this class is extensible because it is not marked as final. But Kotlin classes are final by default, and classes must be marked with the keyword open for it to be extensible. The PizzaMaker class above is implicitly final. If we try to subclass it using the code below,

    class PizzaMakerChild: PizzaMaker() {}

then the compiler will complain that,

    This type is final, so it cannot be inherited from
Extension Function

Even if the class is final, we can still use extension functions to extend it. An extension function syntax is similar to a regular function. The only extra requirement is that we have to provide a Receiver type, which goes right before the function name and a period(.).

    fun PizzaMaker.makeSomeOtherPizza() {}

In the above function, PizzaMaker is the Receiver type.

Using the syntax above, let us create two extension functions of PizzaMaker using the code below. They are top-level functions, so make sure you put them outside of the PizzaMaker class.

    fun PizzaMaker.makeVeganPizza() {
       println("Vegan Pizza")
    }

    fun PizzaMaker.makeMeatLoverPizza(){
       println("Meat Lover")
    }

Now, we can also call makeVeganPizza() and makeMeatLoverPizza() directly from any PizzaMaker reference. In main(), we call the code below.

    fun main(){
       val maker = PizzaMaker()

       maker.makePepperoniPizza()
       maker.makeHawaiianPizza()

       maker.makeVeganPizza() //extensions
       maker.makeMeatLoverPizza() //extensions
    }

And the output is:

    Pepperoni Pizza
    Hawaiian Pizza
    Vegan Pizza
    Meat Lover
Extension Properties

Besides being to add extension functions, we can also add extension properties. When we add an extension property, we do not actually add a new field into the class, so this property does not have a backing field(it cannot store its own state by itself). So for this extension property, we must define a getter and an optional setter.

Suppose we add a field to count the number of pizzas made each time a make*() function is called (the count goes up for extension functions, too).

    class PizzaMaker() {

       var count: Int = 0

       fun makePepperoniPizza(){
           println("Pepperoni Pizza")
           count++
       }

       fun makeHawaiianPizza(){
           println("Hawaiian Pizza")
           count++
       }
    }

    //class PizzaMakerChild: PizzaMaker() {} //illegal

    fun PizzaMaker.makeVeganPizza() {
       println("Vegan Pizza")
       count++
    }

    fun PizzaMaker.makeMeatLoverPizza(){
       println("Meat Lover")
       count++
    }

And then we can add an extension property to store the amount of revenue made ($8 for each pizza) using the code below.

    val PizzaMaker.revenue: Int //$8 for each pizza
       get() = count * 8

With the extension property above, we can also access (get) it directly via a PizzaMaker reference.

If we call this code in main(),

    fun main(){
       val maker = PizzaMaker()

       maker.makePepperoniPizza()
       maker.makeHawaiianPizza()

       maker.makeVeganPizza() //extensions
       maker.makeMeatLoverPizza() //extensions

       println(maker.count) //internal property
       println(maker.revenue) //extension property
    }

We now can access the revenue property directly from the PizzaMaker reference. The code output:

    Pepperoni Pizza
    Hawaiian Pizza
    Vegan Pizza
    Meat Lover
    4
    32

We made a total of 4 pizzas, so the revenue is $32.

Solution Code
    package com.example

    fun main(){
       val maker = PizzaMaker()

       maker.makePepperoniPizza()
       maker.makeHawaiianPizza()

       maker.makeVeganPizza() //extensions
       maker.makeMeatLoverPizza() //extensions

       println(maker.count) //internal property
       println(maker.revenue) //extension property
    }

    class PizzaMaker() {

       var count: Int = 0

       fun makePepperoniPizza(){
           println("Pepperoni Pizza")
           count++
       }

       fun makeHawaiianPizza(){
           println("Hawaiian Pizza")
           count++
       }
    }

    //class PizzaMakerChild: PizzaMaker() {} //illegal

    fun PizzaMaker.makeVeganPizza() {
       println("Vegan Pizza")
       count++
    }

    fun PizzaMaker.makeMeatLoverPizza(){
       println("Meat Lover")
       count++
    }

    val PizzaMaker.revenue: Int //$8 for each pizza
       get() = count * 8
Summary

We have learned how to create our own extension functions and extension properties. The fill project code can be found here https://github.com/dmitrilc/DaniwebKotlinExtensionFunctions/tree/master

How to the async() coroutine builder in Kotlin

Introduction

launch() and async() are two of the most common coroutine builders to use in Kotlin, but they are somewhat different in usage.

launch() returns a Job object, which can be used to cancel or perform other operations on the underlying coroutine. If our coroutine lambda returned a useful result, then there is no way to access it via this Job object.

async() is different to launch() because it returns a Deferred<T> object, where T stands for the result returned from its lambda expression. Deferred<T> is actually a child of Job, and via a Deferred object, we can retrieve the underlying lambda value.

In this tutorial, we will learn how to use the async() coroutine builder.

Goals

At the end of the tutorial, you would have learned:

  1. How to use the async() coroutine builder.
Prerequisite Knowledge
  1. Intermediate Kotlin.
  2. Basic coroutine.
Tools Required
  1. A Kotlin IDE such as IntelliJ Community Edition.
Project Setup

To follow along with the tutorial, perform the steps below:

  1. Create a new Kotlin project, using JDK 16 as the project JDK and Gradle 7.2 as the build tool.

  2. Copy and paste the configuration code below into your build.gradle.kts file.

     import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
    
     plugins {
        kotlin("jvm") version "1.5.31"
     }
    
     group = "com.example"
     version = "1.0-SNAPSHOT"
    
     repositories {
        mavenCentral()
     }
    
     dependencies {
        implementation ("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.5.2")
        testImplementation("org.jetbrains.kotlin:kotlin-test:1.5.31")
     }
    
     tasks.test {
        useJUnitPlatform()
     }
    
     tasks.withType<KotlinCompile>() {
        kotlinOptions.jvmTarget = "11"
     }
  3. Under src/main/kotlin, create a new package called com.example.

  4. Create a new Kotlin file called Entry.kt.

  5. Create the main() function inside this file.

async() Concept Overview

In order to understand how the coroutine builder async() works, let us look at its function signature:

    fun <T> CoroutineScope.async(
    context: CoroutineContext = EmptyCoroutineContext,
    start: CoroutineStart = CoroutineStart.DEFAULT,
    block: suspend CoroutineScope.() -> T)
    : Deferred<T>

From the method signature above, we can see that it takes 3 arguments, with the first and the second arguments being optional arguments, so we are only required to provide the last argument, which is a lambda that returns a type of T. The returned value will be wrapped inside the Deferred<T> object that the async() function returns(the return value from the block lambda expression is different from the return value of the async() function).

Compared to the launch() coroutine builder, with the async() builder, we are only mostly interested in the return value of the lambda and the Deferred<T> object. Because Deferred is a child of Job, it provides extra functionalities.

async() Usage

Because async() does not return the underlying value directly to us, we will have to get it from the Deferred object. Copy and paste the top-level function asyncVerbose() below into your Entry.kt file.

    suspend fun asyncVerbose() = coroutineScope { //1
       val deferred: Deferred<String> = async { //2
           println("waiting") //3
           delay(2000L) //4
           "result" //5
       }

       val result: String = deferred.await() //6

       println(result) //7
    }

The code above is purposely verbose to demonstrate how the underlying value is retrieved.

  1. async() must be used inside a coroutine scope, so we created a dedicated coroutine scope for it to run in at line 1.
  2. Line 2 is where we call the async(). We only have to provide the last argument, which is a lambda expression that returns some type, so the passing-trailing-lambda syntax is used here.
  3. Line 3 prints something to the console to indicate that the coroutine has started.
  4. Line 4 waits for 2 seconds, pretending that the coroutine is performing some long-running operations.
  5. Line 5 returns the result of the lambda, which is just a simple String.
  6. On line 6, we call await() on the Deferred object received from async(). await() does not block the current thread. It only pauses the coroutine until the calculation is complete. After the calculation is completed, wait() returns a result.

We can call the method in main() like this:

    fun main() = runBlocking {
       asyncVerbose()
    }

The code completes in about 2 seconds and prints:

    waiting
    result
Retrieve the result without waiting

Besides await(), it is also possible to attempt to retrieve the result immediately. For tasks that you do not want to wait for forever, you can use the function getCompleted(). This function will throw a runtime exception if the task has not been completed yet. Whether to catch this exception and log/rethrow/handle it is your choice.

Let us add another function called asyncNoWait() that will use the getCompleted() function. Copy and paste the code below into Entry.kt.

    @OptIn(ExperimentalCoroutinesApi::class) //8
    suspend fun asyncNoWait() = coroutineScope { //9
       val deferred: Deferred<String> = async {
           println("waiting")
           delay(5000L)
           "result"
       }

       delay(2000L) //10
       print("waited for too long, timing out...")

       val result: String = deferred.getCompleted() //11

       println(result)
    }

In the code snippet above, the async lambda is actually waiting for 5 seconds. At line 10, the application is only waiting for 2 seconds before giving up and moving on to call getCompleted(). Because the async() lambda has not completed yet, the code throws an IllegalStateException:

    waiting
    waited for too long, timing out...Exception in thread "main" java.lang.IllegalStateException: This job has not completed yet
        at kotlinx.coroutines.JobSupport.getCompletedInternal$kotlinx_coroutines_core(JobSupport.kt:1199)
        at kotlinx.coroutines.DeferredCoroutine.getCompleted(Builders.common.kt:100)
        at com.example.EntryKt$asyncNoWait$2.invokeSuspend(Entry.kt:33)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:234)
        at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:166)
        at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:397)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:431)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl$default(CancellableContinuationImpl.kt:420)
        at kotlinx.coroutines.CancellableContinuationImpl.resumeUndispatched(CancellableContinuationImpl.kt:518)
        at kotlinx.coroutines.EventLoopImplBase$DelayedResumeTask.run(EventLoop.common.kt:489)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at com.example.EntryKt.main(Entry.kt:5)
        at com.example.EntryKt.main(Entry.kt)
Solution Code
    package com.example

    import kotlinx.coroutines.*

    fun main() = runBlocking {
       //asyncVerbose()
       asyncNoWait()
    }

    suspend fun asyncVerbose() = coroutineScope { //1
       val deferred: Deferred<String> = async { //2
           println("waiting") //3
           delay(2000L) //4
           "result" //5
       }

       val result: String = deferred.await() //6

       println(result) //7
    }

    @OptIn(ExperimentalCoroutinesApi::class) //8
    suspend fun asyncNoWait() = coroutineScope { //9
       val deferred: Deferred<String> = async {
           println("waiting")
           delay(5000L)
           "result"
       }

       delay(2000L) //10
       print("waited for too long, timing out...")

       val result: String = deferred.getCompleted() //11

       println(result)
    }
Summary

We have learned how to use the async() coroutine builder. It should be used whenever we expect a value from the lambda expression.

The full project code can be found here https://github.com/dmitrilc/DaniwebCoroutineAsyncKotlin/tree/master

Combine RxJava Observables – mergeWith() and concatWith()

Introduction

In this tutorial, we will look at the RxJava mergeWith() and concatWith() operators. Amongst others, they are used to combine Observables together.

All of the RxJava operators have ambiguous names, and behave differently, so let us take a look at 2 operators, mergeWith() and concatWith(), to see how they are different.

Goals

At the end of the tutorial, you would have learned:

  1. How to use the RxJava mergeWith() operator.
  2. How to use the RxJava concatWith() operator.
Prerequisite Knowledge
  1. Intermediate Java.
  2. Basic RxJava 3.
Tools Required
  1. A Java IDE such as IntelliJ Community Edition.
  2. The project uses Gradle 7.2 and JDK 17, you can use different versions if you wish.
Project Setup

To follow along with the tutorial, perform the steps below:

  1. Create a new Java Gradle project.

  2. In the build.gradle file, add the dependency for RxJava 3.

     implementation 'io.reactivex.rxjava3:rxjava:3.1.1'
  3. Create a new package called com.example under the src/main folder.

  4. Create a new class called Entry.

  5. Create the main() method inside the Entry class.

mergeWith() Concept Overview

Among all of the RxJava operators, I consider mergeWith() to be one of the easiest combining operators to understand.

mergeWith() is an instance method, so it must be used from a current Observable object. Its method signature is

    public final @NonNull Observable<T> mergeWith(@NonNull ObservableSource<? extends T> other)

Fortunately, its documentation includes a graphic depicting how the two Observable(s) interact with one another when being combined with mergeWith().

mergewtih.png

In the picture above, The top two arrows represent two different Observables. The first Observable emits a red circle; the second Observable emits the yellow circle, but they do not necessarily have to emit their circles in this same order. They do not wait for one another and the mergeWith() operator also does not perform any scheduling.

These two Observables must be of the same Generic bounds, which means you cannot combine an Observable<String> with an Observable<Integer>. If you have not noticed, all 5 values emitted from both Observables are of type Circle, although they might have different color attributes.

mergeWith() also does not perform any transforming of the Observables themselves. It only emits exactly what it received from the Observables downstream.

mergeWith() Usage

In the Entry class, create a mergeWith() method just to encapsulate our example code for the mergeWith() operator. Copy and paste the code below.

    private static void mergeWith(){
       Observable.intervalRange(1, 10, 0, 2000,TimeUnit.MILLISECONDS) //1
               .mergeWith(Observable.intervalRange(11, 10, 1000, 2000, TimeUnit.MILLISECONDS)) //2
               .blockingSubscribe(System.out::println); //3
    }

On line 1, we have created an Observable with intervalRange() to simulate an async data stream starting from 1 to 10 with no starting delay. This first Observable does not have any starting delay because I want it to start emitting first, before the second Observable(which has a delay of 1 second).

The second Observable created on line 2 is also created using intervalRange(). The only difference between the two is that the first value it will emit is 11(to make it easy to discern in the terminal which Observable is emitting later) and the starting delay is 1 second because I want it to start after the first Observable.

After the initial delay, they both emit values at every two seconds. If we call mergeWith(), which is the method that we just created, then we will receive the output below.

    1
    11
    2
    12
    3
    13
    4
    14
    5
    15
    6
    16
    7
    17
    8
    18
    9
    19
    10
    20

I purposely synced the time of the emitted values to make the concept easy to understand. As stated previously, the Observables do not have to wait for one another when emitting values.

The mergeWith() operator has variants that will take other types of arguments besides ObservableSource, such as SingleSource, MaybeSource, and CompletableSource,

    public final @NonNull Observable<T> mergeWith(@NonNull SingleSource<? extends T> other)
    public final @NonNull Observable<T> mergeWith(@NonNull MaybeSource<? extends T> other)
    public final @NonNull Observable<T> mergeWith(@NonNull CompletableSource other)

but the concept behind the operator is the same for all 4 variants of mergeWith().

concatWith() Concept Overview

Now that we have understood mergeWith(), concatWith() should be quite simple to understand. The method signature for concatWith() is:

    public final @NonNull Observable<T> concatWith(@NonNull ObservableSource<? extends T> other)

The flow of concatWith() is depicted in the picture below (from the official docs).

concatWith.png

Similar to mergeWith(), concatWith():

  1. Requires that both Observables extend the same Generic bounds.
  2. Is an instance method.
  3. Does not perform any operation on the emitted values.

But the similarities pretty much stop there, the main difference between concatWith() and mergeWith() is:

  1. The second Observable cannot start until the first Observable is complete. Do you recall that, with mergeWith(), they both start at the same time?
concatWith() Usage

To put the concept that we just learned into code, create a method called concatWith() to encapsulate our example like the code below.

    private static void concatWith(){
       Observable.intervalRange(1, 10, 0, 2000,TimeUnit.MILLISECONDS) //1
               .concatWith(Observable.intervalRange(11, 10, 1000, 2000, TimeUnit.MILLISECONDS)) //2
               .blockingSubscribe(System.out::println); //3
    }

The code snippet above is almost exactly the same as the code used for mergeWith(), with the only difference being that we are using concatWith() instead of mergeWith() on the 2nd line. If we run this method, we will see that the second Observable does not start until the second Observable.

The code outputs

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20

Where 1 to 10 were emitted from the first Observable, and 11 to 20 were emitted by the second Observable.

Solution Code
    package com.example;

    import io.reactivex.rxjava3.core.Observable;

    import java.util.concurrent.TimeUnit;

    public class Entry {
       public static void main(String[] args){
           //mergeWith();
           concatWith();
       }

       private static void mergeWith(){
           Observable.intervalRange(1, 10, 0, 2000,TimeUnit.MILLISECONDS) //1
                   .mergeWith(Observable.intervalRange(11, 10, 1000, 2000, TimeUnit.MILLISECONDS)) //2
                   .blockingSubscribe(System.out::println); //3
       }

       private static void concatWith(){
           Observable.intervalRange(1, 10, 0, 2000,TimeUnit.MILLISECONDS) //1
                   .concatWith(Observable.intervalRange(11, 10, 1000, 2000, TimeUnit.MILLISECONDS)) //2
                   .blockingSubscribe(System.out::println); //3
       }
    }
Summary

We have learned how to use the mergeWith() and concatWith() operators in RxJava. The code snippet can be found here https://github.com/dmitrilc/DaniwebRxJavaMergeWithConcatWith

RxJava 3 – How to create and observe Observables

Introduction

RxJava 3 includes 5 core classes:

  1. Flowable,
  2. Observable,
  3. Single,
  4. Completable,
  5. Maybe.

This tutorial aims to teach the basic concepts behind Observable, which serves as a foundation for understanding the other 4 classes.

Goals

At the end of the tutorial, you would have learned:

  1. How to create an Observable.
  2. How to subscribe to an Observable using an Observer.
  3. How to subscribe to an Observable using Consumers.
Prerequisite Knowledge
  1. Intermediate Java.
  2. Basic knowledge of the Observer pattern.
Tools Required
  1. A Java IDE such as IntelliJ Community Edition.
  2. The project uses Gradle 7.2 and JDK 17, you can use different versions if you wish.
Project Setup

To follow along with the tutorial, perform the steps below:

  1. Create a new Java Gradle project.

  2. In the build.gradle file, add the dependency for RxJava 3.

     implementation 'io.reactivex.rxjava3:rxjava:3.1.1'
  3. Create a new package called com.example under the src/main folder.

  4. Create a new class called Entry.

  5. Create the main() method inside the Entry class.

Creating an Observable

The Observable interface (io.reactivex.rxjava3.core.Observable) has a lot of builder methods that can be used to create an instance of Observable. For this tutorial, we will use the intervalRange() builder to simulate an async stream of data and to keep our code simple.

Below is the method signature of intervalRange():

    public static @NonNull Observable<Long> intervalRange(long start,
            long count,
            long initialDelay,
            long period,
            @NonNull TimeUnit unit)

The Observable from intervalRange() will emit a Long object after each period of waiting. The first parameter is the starting number that we want to emit. Each number after the first will increment by one. Once the amount of numbers emitted is equal to the count parameter, then the Observable will complete.

With the concept out of the way, we add the code into our main() method like below.

    Observable<Long> obs = Observable.intervalRange(1L, 5L, 0L, 2L, TimeUnit.SECONDS);

The line of code above will generate an Observable<Long> that will emit a n+1 number every two seconds to its subscribers.

Subscribing with an Observer

The Observable that we created in the previous section does nothing for now. For it to be useful, we must subscribe to it. One way to subscribe to an Observable is with an Observer.

An Observer (io.reactivex.rxjava3.core.Observer) is an interface that we can subclass to create an instance of. There are 4 methods that we must override:

  1. onSubscribe: The Observable will call this method when the subscription starts.
  2. onNext: The Observable will call this method when it emits a new item.
  3. onError: The Observable will call this method when it throws an exception.
  4. onComplete: The Observable will call this method when it is completed.

To subscribe an Observer to an Observable, we would need to use one of the instance methods from Observable. The method to use in this tutorial is blockingSubscribe() for simplicity. The method signature is:

    public final void blockingSubscribe(@NonNull Observer<? super T> observer)

We can either create a dedicated Observer class and pass its instance into blockingSubscribe(), or we can just create an anonymous class on the fly. Because The official docs tend to use an anonymous class, so we will do the same thing here.

In the Entry class, create a subWithObserver() method using the code snippet below.

    private static <T> void subWithObserver(Observable<T> obs){ //1

       obs.blockingSubscribe(new Observer<>(){ //2
           private Disposable disposable; //3

           @Override
           public void onSubscribe(@NonNull Disposable d) { //4
               this.disposable = d; //5
           }

           @Override
           public void onNext(@NonNull T item) { //6
               System.out.println("Received in Observer: " + item); //7
           }

           @Override
           public void onError(@NonNull Throwable e) {
               disposable.dispose(); //8
               e.printStackTrace();
           }

           @Override
           public void onComplete() {
               disposable.dispose(); //9
               System.out.println("Complete"); //10
           }
       });
    }

The above method takes an Observable argument and then subscribes an anonymous Observer to it. This Observer will print a line every time a new item is emitted. This Observer will also call dispose() on the Disposable object received from the Observable when the Observable is completed.

Now we are ready to call the method in main().

    subWithObserver(obs);

And the output would be:

    Received in Observer: 1
    Received in Observer: 2
    Received in Observer: 3
    Received in Observer: 4
    Received in Observer: 5
    Complete
Subscribing with Consumers

We can also subscribe to an Observable using Consumers, with each representing a callback (as opposed to having to override all 4 methods with Observer). For blockingSubscribe(), the maximum amount of callbacks that we can provide is three: onNext(), onError(), onComplete(). The method signature for this variant of blockingSubscribe() is:

    public final void blockingSubscribe(@NonNull Consumer<? super T> onNext,
            @NonNull Consumer<? super Throwable> onError,
            @NonNull Action onComplete)

As you can see, we only have to provide 2 Consumer(s) and one Action(similar to a Runnable). Create a method subWithConsumers() based on the code below.

    private static <T> void subWithConsumers(Observable<T> obs){
       obs.blockingSubscribe(
               item -> System.out.println("Received in Consumer: " + item),
               error -> error.printStackTrace(),
               () -> System.out.println("Complete")
       );
    }

The method above is almost exactly like the subWithObserver() method in functionality, but with much shorter code. There are also other variants which only require you to provide one or two Consumer(s) if you do not care about the other callbacks.

    public final void blockingSubscribe(@NonNull Consumer<? super T> onNext)
    public final void blockingSubscribe(@NonNull Consumer<? super T> onNext, @NonNull Consumer<? super Throwable> onError)

Finally, we can call the method in main like below.

    //subWithObserver(obs);
    subWithConsumers(obs);

Which will print the exact same thing as the subWithObserver() call:

    Received in Consumer: 1
    Received in Consumer: 2
    Received in Consumer: 3
    Received in Consumer: 4
    Received in Consumer: 5
    Complete
Solution Code
    package com.example;

    import io.reactivex.rxjava3.annotations.NonNull;
    import io.reactivex.rxjava3.core.Observable;
    import io.reactivex.rxjava3.core.Observer;
    import io.reactivex.rxjava3.disposables.Disposable;

    import java.util.concurrent.TimeUnit;

    public class Entry {
       public static void main(String[] args) {
           Observable<Long> obs = Observable.intervalRange(1L, 5L, 0L, 2L, TimeUnit.SECONDS); //1

           //subWithObserver(obs);
           subWithConsumers(obs);
       }

       private static <T> void subWithObserver(Observable<T> obs){ //1

           obs.blockingSubscribe(new Observer<>(){ //2
               private Disposable disposable; //3

               @Override
               public void onSubscribe(@NonNull Disposable d) { //4
                   this.disposable = d; //5
               }

               @Override
               public void onNext(@NonNull T item) { //6
                   System.out.println("Received in Observer: " + item); //7
               }

               @Override
               public void onError(@NonNull Throwable e) {
                   disposable.dispose(); //8
                   e.printStackTrace();
               }

               @Override
               public void onComplete() {
                   disposable.dispose(); //9
                   System.out.println("Complete"); //10
               }
           });
       }

       private static <T> void subWithConsumers(Observable<T> obs){
           obs.blockingSubscribe(
                   item -> System.out.println("Received in Consumer: " + item),
                   error -> error.printStackTrace(),
                   () -> System.out.println("Complete")
           );
       }
    }
Summary

We have learned how to create an Observable and subscribe to it using either an Observer or Consumers. The full project code can be found here https://github.com/dmitrilc/DaniWebRxJavaBuilders

WP Fastest Cache Patches Authenticated SQL Injection and Stored XSS Via CSRF Vulnerabilities

The Jetpack Scan team has published a summary of two issues recently discovered in the WP Fastest Cache plugin – an Authenticated SQL Injection vulnerability and a Stored XSS Via CSRF vulnerability.

“If exploited, the SQL Injection bug could grant attackers access to privileged information from the affected site’s database (e.g., usernames and hashed passwords),” Automattic security research engineer Marc Montpas said. This particular vulnerability can only be exploited on sites where the Classic Editor plugin is both installed and activated.

“Successfully exploiting the CSRF and Stored XSS vulnerability could enable bad actors to perform any action the logged-in administrator they targeted is allowed to do on the targeted site,” Montpas said. He also found that attackers could “abuse some of these options to store rogue Javascript on the affected website.”

WP Fastest Cache is active on more than 1 million WordPress sites, and the plugin also reports 58,322 paid users. Emre Vona, the plugin’s author, patched the vulnerabilities in version 0.9.5, released this week. Jetpack recommends users update as soon as possible, as both vulnerabilities have a high technical impact if exploited.

LogDNA Adds New Control API Suite

LogDNA, a log management solutions provider, has released a new Control API Suite that includes four new APIs dedicated to managing service configurations on the platform. These new APIs allow developers to program applications to handle many of the common maintenance pain points that are normally processed manually.

small issue with INNER JOIN with php and mysql

I have a issue with the code below what I am trying to get the image from gifts table with matching the user_gifts id with the gifts_id but can not seam to get it right anyone god any ideas thanks.

public function get_my_gifts($user_id) {
        global $db;
        $my_gifts = [];
        $get_my_gifts = $db->query(sprintf("SELECT gifts.image FROM users_gifts INNER JOIN gifts ON `users_gifts`.gift_id = `gifts`.gift_id WHERE users_gifts.to_user_id = %s", secure($user_id, 'int') )) or _error("SQL_ERROR_THROWEN");

        if($get_my_gifts->num_rows > 0) {
            while($my_gift = $get_my_gifts->fetch_assoc()) {
                $my_gifts[] = $my_gift;
            }
        }
        return $my_gifts;

}

Yours
Simon

MYSQL Update with Bootstrap Modal not working

Hi, I need your help.

I am a newbie in PHP and my update statement is not working despite seeing all the values from the console.

My code for the bootstrap modal file:

<!----------------------------------- modal begin -------------------------------------->
<div class="modal fade" id="editmodal" tabindex="-1" role="dialog" aria-labelledby="myModalLabel" aria-hidden="true">
  <div class="modal-dialog">
    <div class="modal-content">
    <div class="modal-header">
        <h5 class="modal-title" id="exampleModalLabel">Edit SAB Homes Properties</h5>
          <button type="button" class="close" data-dismiss="modal" aria-label="Close">
           <span aria-hidden="true">&times;</span>
          </button>
</div>
<div class="modal-body">
  <form class="form-sample" form action="updatehouse.php" method="POST">
          <div class="content-wrapper">
          <div class="row">
            <div class="col-12 grid-margin">
              <div class="card">
                <div class="card-body">
                  <h4 class="card-title">Update Property</h4>
                    <div class="form-group">
                      <label for="exampleInputUsername1">House Name</label>
                      <input type="hidden" name="update_id" id="update_id"/>
                      <input type="text" class="form-control" id="house_name" name="house_name" />
                    </div>
                    <div class="form-group">
                      <label for="exampleInputEmail1">Landlord</label>
                      <input type="text" class="form-control" id="house_landlord" name="house_landlord" />
                    </div>
                    <div class="form-group">
                      <label for="exampleInputPassword1">Location</label>
                      <input type="text" class="form-control" id="house_location" name="house_location" />
                    </div>
                    <div class="form-group">
                      <label for="exampleInputConfirmPassword1">Commission</label>
                      <input type="text" class="form-control" id="house_commission" name="house_commission" />
                    </div>
                    <div class="form-group">
                      <label for="exampleInputConfirmPassword1">Service Charge</label>
                      <input type="text" class="form-control" id="house_service_charge" name="house_service_charge" />
                      <input type="hidden" name="house_siku_added" id="house_siku_added"/>
                    </div>
                </div>
            </div>
          </div>
        </div>
      </div>
          <div class="modal-footer">
            <button type="button" class="btn btn-secondary" data-dismiss="modal">Close</button>
            <button type="submit" class="btn btn-primary" name="updatedata" id="updatedata">Save changes</button>
          </div>
          </form>
        </div>
      </div>
    </div>
  </div>

<!----------------------------------- modal end ---------------------------------------->

The table that has the data:

<div class="row">
          <div class="col-lg-12 grid-margin stretch-card">
              <div class="card">
                <div class="card-body">
                  <h4 class="card-title">Houses List</h4>
                  <p class="card-description">&nbsp;</p>
                  <div class="table-responsive">
                    <table class="table table-striped">
                      <thead>
                        <tr>
                          <th width="10%">ID </th>
                          <th width="21%">Property Name</th>
                          <th width="14%">Landlord</th>
                          <th width="14%">Location</th>
                          <th width="14%">Commission</th>
                          <th width="14%">Service Charge</th>
                          <th width="14%">Registered</th>
                          <th width="22%">Edit</th>
                        </tr>
                      </thead>
                      <tbody>
                        <?php
                          $house = $connection->query("SELECT * FROM houses");
                          while($row=$house->fetch_assoc()):?>  
                          <tr>
                          <td><?php echo $row['house_id']; ?></td>  
                          <td><?php echo $row['house_name']; ?></td>
                                    <td><?php echo $row['house_landlord']; ?></td>
                          <td><?php echo $row['house_location']; ?></td>
                          <td><?php echo $row['house_commission']; ?></td>
                          <td><?php echo $row['house_service_charge']; ?></td>
                          <td><?php echo $row['house_siku_added']; ?></td> 
                          <td><button type="button" class="btn btn-success editbtn">Edit House</button></td>
                       </tr>
                                  <?php endwhile; ?>
                    </tbody>
                    </table>
                     <?php 
                          $pr_query = "select * from houses ";
                          $pr_result = mysqli_query($connection,$pr_query);
                          $total_record = mysqli_num_rows($pr_result );
                          $total_page = ceil($total_record/$num_per_page);
                          if($page>1)
                          {
                              echo "<a href='listhouses.php?page=".($page-1)."' class='btn btn-danger'>Previous</a>";
                          }
                          for($i=1;$i<$total_page;$i++)
                          {
                              echo "<a href='listhouses.php?page=".$i."' class='btn btn-primary'>$i</a>";
                          }
                          if($i>$page)
                          {
                              echo "<a href='listhouses.php?page=".($page+1)."' class='btn btn-danger'>Next</a>";
                          }
                    ?>
                  </div>
                </div>
              </div>
            </div>
        </div>

The script used to open the modal.

 <script>
  $(document).ready(function() {
    $('.editbtn').on('click', function(){
      $('#editmodal').modal('show');
      $tr = $(this).closest('tr');
    var data = $tr.children("td").map(function(){
      return $(this).text();
    }).get();
    console.log(data);
    $('#update_id').val(data[0]);
    $('#house_name').val(data[1]);
    $('#house_landlord').val(data[2]);
    $('#house_location').val(data[3]);
    $('#house_commission').val(data[4]);
    $('#house_service_charge').val(data[5]);
    $('#house_siku_added').val(data[6]);
    });
  });   
</script>

This is the data passed:

Screenshot_2021-10-14_at_15_39_24.png

The php file:

<?php
include('config/db_connect.php');
if(isset($_POST['updatedata']))
    {   
                $house_id = mysqli_real_escape_string($connection, $_REQUEST['$update_id']);
                $house_name = mysqli_real_escape_string($connection,$_REQUEST['$house_name']);
                $house_landlord = mysqli_real_escape_string($connection, $_REQUEST['$house_landlord']);
                $house_location = mysqli_real_escape_string($connection, $_REQUEST['$house_location']);
                $house_commission = mysqli_real_escape_string($connection, $_REQUEST['$house_commission']);
                $house_service_charge = mysqli_real_escape_string($connection, $_REQUEST['$house_service_charge']);
                $house_siku_added = mysqli_real_escape_string($connection, $_REQUEST['$house_siku_added']);

    $sql = "UPDATE houses SET house_name = '$house_name', house_landlord = '$house_landlord', house_location = '$house_location', house_commission = '$house_commission', house_service_charge = '$house_service_charge', house_siku_added = now() WHERE house_id = '$house_id'";   
        $sqlQuery = mysqli_query($connection, $sql);

                    if($sqlQuery){
                     echo '<script> alert("Record was updated successfully."); </script>';
                     header("Location:listhouses.php");
                        }
                    else
                        {
                        echo '<script> alert("Data not updated"); </script>';
                        }
    }       
?>

I would really appreciate if someone would help me see where I am going wrong with the update statement. Thank you.

Collective #683







Collective 683 item image

Kuma

Kuma is a self-hosted monitoring tool similar to “Uptime Robot” with a great UI. Learn more about it in this video.

Check it out


Collective 683 item image

YoHa

YoHa is a hand tracking engine that is built with the goal of being a versatile solution in practical scenarios where hand tracking is employed to add value to an application.

Check it out



Collective 683 item image

Rowy

A free Airtable alternative: Manage Firestore data in a spreadsheet-like UI, write Cloud Functions effortlessly in the browser, and connect to third party platforms.

Check it out



Collective 683 item image

Huetone

Use LCH color space to come up with predictable and accessible color palettes.

Check it out


Collective 683 item image

Flowrift

A growing library filled with Tailwind CSS UI blocks, components and templates.

Check it out





Collective 683 item image

Bor

A user-friendly, tiny source code searcher written by pure Python.

Check it out







Collective 683 item image

3dicons

Some months ago, Vijay Verma shared 60+ free and open-source 3D icons. This collection has now grown to over a thousand icons! Worth checking out again.

Check it out

The post Collective #683 appeared first on Codrops.

visual basic login system

hello i am new to the coding world and i am trying to get this login code to only accept the names and passwords in the access database, current any can login. please assist

Imports System.Data
Imports System.Data.OleDb
Imports System.Data.DataTable
Public Class Form1

Private Sub btnLogin_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnLogin.Click
    Dim conn As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Users\Katlego\Documents\logindb.accdb")
    Dim cmd As New OleDbCommand("select * from logintable where user1=@user1 and pass1=@pass1", conn)
    cmd.Parameters.Add("@user1", OleDbType.VarChar).Value = txtUsername.Text
    cmd.Parameters.Add("@user1", OleDbType.VarChar).Value = txtPassword.Text
    Dim adapter1 As New OleDbDataAdapter(cmd)
    Dim logintable As New DataTable
    adapter1.Fill(logintable)
    If logintable.Rows.Count <=
        0 Then
        MsgBox("error username or password")
    Else
        MsgBox("login successfull")
    End If
End Sub

Private Sub btnCancel_Click(sender As Object, e As EventArgs) Handles btnCancel.Click
    Me.Close()
End Sub

End Class

How to Exclude Pages from WordPress Search Results (Step by Step)

Do you want to learn how to exclude pages from the WordPress search results?

By default, the internal search on your WordPress site will include all posts and pages in the search results. To make it easier for your site visitors to find what they’re looking for, you may want to exclude certain content, so it never appears in those results.

In this article, we’ll show you how to exclude certain pages from the WordPress search results, step by step.

How to exclude pages from WordPress search results (step by step)

Why Exclude Pages from WordPress Search Results?

The default WordPress site search includes all of your posts, pages, and custom post types in the search results. This works well enough for many WordPress website owners.

However, if you’re running an online store, then you may want to hide pages like your account page, checkout page, thank you page, and more.

The same goes for those running membership websites or online courses. You may want to hide subscriber-only content from appearing in search.

With that said, let’s show you how to exclude pages from your sitewide WordPress search results.

Note: This guide covers on-site search results, when a visitor to your website uses a search box to search through your website content. If you’re looking to exclude pages from Google search results, then you should see our guide on how to hide a WordPress page from Google.

Create a New WordPress Search Algorithm with a WordPress Plugin

The easiest way to exclude pages from the WordPress search results is by using the SearchWP plugin. It’s the best search plugin for WordPress used by over 30,000 websites.

It’s very easy to use and gives you complete control of your WordPress search results.

First thing you need to do is install and activate the plugin. For more details, see our step by step guide on how to install a WordPress plugin.

Upon activation, navigate to Settings » SearchWP and then click the ‘License’ menu option.

Enter SearchWP license key

Then, you can enter your license key into the ‘License’ box and click the ‘Activate’ button. You can find this information in your account on the SearchWP website.

After that, you need to click on the ‘Engines’ menu option.

SearchWP default engine settings

This takes you to a page where you need to set your default search engine settings. Think of it like creating your own Google search algorithm that’s only for your website.

With SearchWP, you need to create a sitewide search engine before you can start excluding pages from the WordPress search results.

You can change the search engine settings by clicking on the ‘Posts’, ‘Pages’, ‘Media’, and other custom post type drop downs.

Each section has a weight multiplier that will change how the search engine ranks your website’s content.

For example, if you move the ‘Title’ slider down and the ‘Content’ slider up, then the content will hold more value than the title.

Customize search engine weight

You can also completely remove all pages from appearing in the WordPress search here.

Simply click the ‘Remove’ button in the ‘Pages’ section.

Click button to remove page source

This brings up a popup where you need to uncheck the ‘Pages’ box.

Then, click the ‘Done’ button to remove all pages from appearing in search.

Uncheck pages box and save

After you’re done customizing the default search engine settings, you need to click the ‘Save Engines’ button to create your first engine.

Save default SearchWP engine

Exclude Specific Pages From the WordPress Search Results

If you only want to exclude specific pages from the search results on your website, then this section is for you.

To do this, we’ll be using the Exclude UI extension from SearchWP. When you’re logged into your SearchWP account, go to the Exclude UI download page and click ‘Download Extension’.

SearchWP exclude UI extension

Then, install and activate the extension as you would any other WordPress plugin.

After that, open up the page that you want to exclude from the search results.

Next, make sure the ‘Page’ option is selected in your right hand menu, and simply check the ‘Exclude from search’ checkbox.

Exclude from search checkbox

Then click the ‘Update’ button, and that page will now be
hidden from the search results. Even if your visitors enter the exact title of the page into your search bar, it will not appear.

If you want to hide more individual pages from the search results, then simply follow the above steps again.

To exclude specific blog posts from WordPress search results, you can follow the same steps above. The only difference is that the right-hand menu will say ‘Post’ instead of ‘Page’.

We hope this article helped you learn how to exclude pages from the WordPress search results. You may also want to see our beginner’s guide on how to get a free SSL certificate for your website and our best business phone services for small businesses.

If you liked this article, then please subscribe to our YouTube Channel for WordPress video tutorials. You can also find us on Twitter and Facebook.

The post How to Exclude Pages from WordPress Search Results (Step by Step) appeared first on WPBeginner.