SW Design: FB open sources LogDevice

https://www.infoq.com/news/2018/09/logdevice-distributed-logstorage

About how FB uses it and the design features it was made to support (quoting):
"Facebook open sourced their internal distributed log storage project called LogDevice. It offers high write availability using replication, durable log storage and recovery from failure.

Most of Facebook's applications that perform logging require high write availability, durable storage of logs, and workloads that vary in terms of performance and latency requirements. Another important requirement was to be able to survive hardware failures. An older Facebook project called Scribe was more focused on aggregating logs to central storage, and there were cases where data loss could occur. Scribe now uses LogDevice as a log storage backend.

Facebook uses LogDevice internally in its datacenters for stream processing pipelines, distribution of database index updates, machine learning pipelines, replication pipelines, and durable task queues where it ingests over 1TB/sec of data. Although Facebook has built a lot of open source tools to manage LogDevice clusters, they are yet to open source any of those except a basic toolset at this point. The LDShell tool allows cluster management from the command line, and the complementary LDQuery command can be used to view cluster statistics."

(...) "[Other] log storage systems [include] Apache BookKeeper and Apache Kafka. 

JAVA: Java 11 feature list

https://www.infoq.com/news/2018/09/java11-released

New things (and deprecated things), quoting from above:
"The feature list for the new version has only evolved modestly since InfoQ reported on this earlier in the year, and the major new features in Java 11 are:

Nest-based access controls (aka "Nestmates"): revisits the implementation of inner classes and eliminates the need for compilers to insert bridge methods.

Dynamic class-file constants (aka "condy"): reduces the cost and disruption of creating new forms of materializable class-file constants and opens the door to new performance and platform approaches.

ZGC (Experimental): a brand-new garbage collector designed for sub-10ms pause times (even on large heaps) with an aim of no more than a 15% performance penalty.

Flight Recorder: low overhead data collection framework for troubleshooting Java applications and the HotSpot JVM, previously a commercial feature, now open-sourced.

New standard HTTP library: A new module java.net.http that standardizes the incubating API and allows flow-based HTTP/1.1 and HTTP/2 support

TLS 1.3: Implement the recently standardized new version of Transport Layer Security

Local-Variable Syntax for Lambda Parameters: Updating the lambda syntax to use the var type-inferencing introduced in Java 10.

The new release also removes some older features to try to streamline the core Java product:

Java EE and CORBA Modules: Require applications to explicitly depend upon EE and CORBA modules if they require them.

Web Start: This feature has been removed with no clear replacement.

Applets: These have been heavily deprecated for some time and are finally removed.

JavaFX: The FX libraries have moved to the OpenJFX project and are removed from the core.

In particular, the EE modules contain the support for JAXB and SOAP - both components that are still in relatively widespread use in many enterprises. Development teams that upgrade to the new version should check carefully whether they will need to modify their build scripts to take these changes into account."

JAVA: Using Java 11 In Production - Things To Know

Java 11 is out. What's new? You pay. Well, this is not all that's new, but you do get to pay if you the Oracle JDK in production: 

• Using Java 11 In Production: Important Things To Know | IntelliJ IDEA Blog

Quoting:

"The new six-monthly release cadence and Oracle’s changes in licensing and support model mean that any organization that deploys a Java application should take this opportunity to look at:

Where they get their JDK from (e.g. Azul, IBM, Oracle, Red Hat, or other OpenJDK build)

Which versions of Java they use in production

Whether or not they expect to get updates (e.g. patches and security fixes) for these versions of Java

Which versions they want to use in the future

How frequently they want to update their version of Java (i.e. to the next major version)

What sort of support (i.e. triage and bug reports) they expect for their version

Changes to the Oracle JDK

In the past, many of us simply downloaded Oracle’s JDK and used this in development, in testing, and in production.  From Java 11, Oracle has changed the license of their JDK, so instead of having a single JDK build which can be used either commercially (i.e. with paid support) or for free (which many of us were doing), they now have two different JDK builds:

Oracle’s JDK (commercial) – you can use this in development and testing for free, but if you use it in production you have to pay for it

Oracle’s OpenJDK (open source) – you can use this for free in any environment, like any open source library

Note that since Java 11, Oracle’s commercial JDK and Oracle’s OpenJDK builds are functionally the same, so we should be able to run our applications on either without having to make any changes or losing any features."

And the components of the JDK are:

• http://cs-fundamentals.com/tech-interview/java/components-of-jdk-environment.php

PS: Kudos to N. Raimundo for pointing this out.

SPACE: TESS architecture and SW image processing made

Due to bandwidth limitations, heavy processing of the high res images taken is made at space and only relevant results are sent back. Here is an interesting description of the new NASA's TESS mission (now that the Kepler mission is being phased out due to lack of fuel) that has the goal of finding new exoplanets:
https://arstechnica.com/science/2018/09/get-ready-for-a-flood-of-new-exoplanets-tess-has-already-spotted-two/

Quoting:
"TESS images a single area for roughly a month before moving on to the next. Over the course of a year, this will allow it to capture most of the sky in a single hemisphere; it will switch to the other hemisphere for its second year of observations. Should the hardware still be operational at the two-year mark, it will have imaged most of the sky, and a similar cycle will likely start again.

This cadence creates some trade offs. If a planet's orbit is such that it doesn't pass in front of its star during the month TESS happens to be pointing that way, we'll miss it (unless it's part of the small overlap between separate areas). This will bias us toward finding planets with short orbital periods, where a transit is guaranteed to happen whenever TESS gets around to pointing at it. Short enough orbits mean we can observe multiple transits during that month, confirming the planet's existence without the need for follow-on observations."

REUSE: Cryptographic library from Google

Quoting:
"Tink is a multi-language, cross-platform cryptographic library developed by a group of cryptographers and security engineers at Google to help developers implement cryptography correctly without being cryptographic experts. Under development for the last two years, version 1.2 adds support for Cloud, Android, and iOS platforms, and C++ and Objective-C."
More at:
https://www.infoq.com/news/2018/09/google-tink-cryto-ios-android

Current implementation details:
"Tink currently provides four cryptographic operations, implemented by specific primitives:

authenticated encryption with associated data (primitive: AEAD)

message authentication codes (primitive: MAC),

digital signatures (primitives: PublicKeySign and PublicKeyVerify)

hybrid encryption (primitives: HybridEncrypt and HybridDecrypt).

Each operation has an associated set of minimal properties and a guarantee. A primitive can have multiple implementations and the user chooses the actual implementation to use by instantiating a key of a corresponding type. (...)

In addition to cryptographic operations, Tink provides support for key management, including key versioning, key rotation, and support for remote key management systems (KMS). Currently, the following KMSes are supported:

Google Cloud KMS

Amazon KMS

Android Keystore

Apple iOS KeyChain (planned)

Tink currently supports Java, C++, and Objective-C. Support for C#, Go, and JavaScript/Node is planned for version 1.3, tentatively scheduled for December 2018."