标签 - hackernews

文章 feed - 评论 feed

2014-02-15

Comments: "" URL: http://www.darpa.mil/opencatalog/

Comments: ""

URL: http://www.darpa.mil/opencatalog/


Aptima Inc. Network Query by Example Analytics 2014-07 https://github.com/Aptima/pattern-matching.git stats Hadoop MapReduce-over-Hive based implementation of network query by example utilizing attributed network pattern matching. ALv2 Boeing/Pitt
Publications SMILE-WIDE: A scalable Bayesian network library Analytics 2014-07 https://github.com/SmileWide/main.git stats SMILE-WIDE is a scalable Bayesian network library. Initially, it is a version of the SMILE library, as in SMILE With Integrated Distributed Execution. The general approach has been to provide an API similar to the existing API SMILE developers use to build "local," single-threaded applications. However, we provide "vectorized" operations that hide a Hadoop-distributed implementation. Apart from invoking a few idioms like generic Hadoop command line argument parsing, these appear to the developer as if they were executed locally. ALv2 Carnegie Mellon University
Publications Support Distribution Machines Analytics 2014-07 https://github.com/dougalsutherland/py-sdm.git stats Python implementation of the nonparametric divergence estimators described by Barnabas Poczos, Liang Xiong, Jeff Schneider (2011). Nonparametric divergence estimation with applications to machine learning on distributions. Uncertainty in Artificial Intelligence. ( http://autonlab.org/autonweb/20287.html ) and also their use in support vector machines, as described by Dougal J. Sutherland, Liang Xiong, Barnabas Poczos, Jeff Schneider (2012). Kernels on Sample Sets via Nonparametric Divergence Estimates. ( http://arxiv.org/abs/1202.0302 ). BSD Continuum Analytics Blaze Infrastructure 2014-07 https://github.com/ContinuumIO/blaze.git stats Blaze is the next-generation of NumPy. It is designed as a foundational set of abstractions on which to build out-of-core and distributed algorithms over a wide variety of data sources and to extend the structure of NumPy itself. Blaze allows easy composition of low level computation kernels (C, Fortran, Numba) to form complex data transformations on large datasets. In Blaze, computations are described in a high-level language (Python) but executed on a low-level runtime (outside of Python), enabling the easy mapping of high-level expertise to data without sacrificing low-level performance. Blaze aims to bring Python and NumPy into the massively-multicore arena, allowing it to leverage many CPU and GPU cores across computers, virtual machines and cloud services. BSD Continuum Analytics Numba Infrastructure 2014-07 https://github.com/numba/numba.git stats Numba is an Open Source NumPy-aware optimizing compiler for Python sponsored by Continuum Analytics, Inc. It uses the LLVM compiler infrastructure to compile Python syntax to machine code.

It is aware of NumPy arrays as typed memory regions and so can speed-up code using NumPy arrays. Other, less well-typed code is translated to Python C-API calls effectively removing the "interpreter" but not removing the dynamic indirection.

Numba is also not a tracing just in time (JIT) compiler. It compiles your code before it runs either using run-time type information or type information you provide in the decorator.

Numba is a mechanism for producing machine code from Python syntax and typed data structures such as those that exist in NumPy.

BSD Continuum Analytics Bokeh Visualization 2014-07 https://github.com/ContinuumIO/bokeh.git stats Bokeh (pronounced bo-Kay or bo-Kuh) is a Python interactive visualization library for large datasets that natively uses the latest web technologies. Its goal is to provide elegant, concise construction of novel graphics in the style of Protovis/D3, while delivering high-performance interactivity over large data to thin clients. BSD Continuum Analytics and Indiana University
Publications Abstract Rendering Visualization 2014-07 https://github.com/JosephCottam/AbstractRendering.git stats Information visualization rests on the idea that a meaningful relationship can be drawn between pixels and data. This is most often mediated by geometric entities (such as circles, squares and text) but always involves pixels eventually to display. In most systems, the pixels are tucked away under levels of abstraction in the rendering system. Abstract Rendering takes the opposite approach: expose the pixels and gain powerful pixel-level control. This pixel-level power is a complement to many existing visualization techniques. It is an elaboration on rendering, not an analytic or projection step, so it can be used as an epilogue to many existing techniques. In standard rendering, geometric objects are projected to an image and represented on that image's discrete pixels. The source space is an abstract canvas that contains logically continuous geometric primitives and the target space is an image that contains discrete colors. Abstract Rendering fits between these two states. It introduces a discretization of the data at the pixel-level, but not necessarily all the way to colors. This enables many pixel-level concerns to be efficiently and concisely captured. BSD Continuum Analytics CDX Visualization 2014-07 https://github.com/ContinuumIO/cdx.git stats Software to visualize the structure of large or complex datasets / produce guides that help users or algorithms gauge the quality of various kinds of graphs & plots. BSD Continuum Analytics and Indiana University
Publications Stencil Visualization 2014-07 https://github.com/JosephCottam/Stencil.git stats Stencil is a grammar-based approach to visualization specification at a higher-level. BSD Data Tactics Corporation Vowpal Wabbit Analytics 2014-07 https://github.com/JohnLangford/vowpal_wabbit.git stats The Vowpal Wabbit (VW) project is a fast out-of-core learning system sponsored by Microsoft Research and (previously) Yahoo! Research. Support is available through the mailing list. There are two ways to have a fast learning algorithm: (a) start with a slow algorithm and speed it up, or (b) build an intrinsically fast learning algorithm. This project is about approach (b), and it's reached a state where it may be useful to others as a platform for research and experimentation. There are several optimization algorithms available with the baseline being sparse gradient descent (GD) on a loss function (several are available). The code should be easily usable. Its only external dependence is on the boost library, which is often installed by default. BSD Data Tactics Corporation Circuit Infrastructure 2014-07 https://code.google.com/p/gocircuit/source/checkout Go Circuit reduces the human development and sustenance costs of complex massively-scaled systems nearly to the level of their single-process counterparts. It is a combination of proven ideas from the Erlang ecosystem of distributed embedded devices and Go's ecosystem of Internet application development. Go Circuit extends the reach of Go's linguistic environment to multi-host/multi-process applications. ALv2 Georgia Tech / GTRI
Publications libNMF: a high-performance library for nonnegative matrix factorization and hierarchical clustering Analytics 2014-07 Pending LibNMF is a high-performance, parallel library for nonnegative matrix factorization on both dense and sparse matrices written in C++. Implementations of several different NMF algorithms are provided, including multiplicative updating, hierarchical alternating least squares, nonnegative least squares with block principal pivoting, and a new rank2 algorithm. The library provides an implementation of hierarchical clustering based on the rank2 NMF algorithm. ALv2 IBM Research
Publications SKYLARK: Randomized Numerical Linear Algebra and ML Analytics 2014-07 2014-05-15 SKYLARK implements Numerical Linear Algebra (NLA) kernels based on sketching for distributed computing platforms. Sketching reduces dimensionality through randomization, and includes Johnson-Lindenstrauss random projection (JL); a faster version of JL based on fast transform techniques; sparse techniques that can be applied in time proportional to the number of nonzero matrix entries; and methods for approximating kernel functions and Gram matrices arising in nonlinear statistical modeling problems. We have a library of such sketching techniques, built using MPI in C++ and callable from Python, and are applying the library to regression, low-rank approximation, and kernel-based machine learning tasks, among other problems. ALv2 Institute for Creative Technologies / USC Immersive Body-Based Interactions Visualization 2014-07 http://code.google.com/p/svnmimir/source/checkout stats Provides innovative interaction techniques to address human-computer interaction challenges posed by Big Data. Examples include:
* Wiggle Interaction Technique: user induced motion to speed visual search.
* Immersive Tablet Based Viewers: low cost 3D virtual reality fly-through's of data sets.
* Multi-touch interfaces: browsing/querying multi-attribute and geospatial data, hosted by SOLR.
* Tablet based visualization controller: eye-free rapid interaction with visualizations. ALv2 Johns Hopkins University
Publications igraph Analytics 2014-07 https://github.com/igraph/xdata-igraph.git stats igraph provides a fast generation of large graphs, fast approximate computation of local graph invariants, fast parallelizable graph embedding. API and Web-service for batch processing graphs across formats. GPLv2 Trifacta (Stanford, University of Washington, Kitware, Inc. Team) Vega Visualization 2014-07 https://github.com/trifacta/vega.git stats Vega is a visualization grammar, a declarative format for creating and saving visualization designs. With Vega you can describe data visualizations in a JSON format, and generate interactive views using either HTML5 Canvas or SVG. BSD Kitware, Inc. Tangelo Visualization 2014-07 https://github.com/Kitware/tangelo.git stats Tangelo provides a flexible HTML5 web server architecture that cleanly separates your web applications (pure Javascript, HTML, and CSS) and web services (pure Python). This software is bundled with some great tools to get you started. ALv2 Harvard and Kitware, Inc.
Publications LineUp Visualization 2014-07 https://github.com/Caleydo/org.caleydo.vis.lineup.demos.git stats LineUp is a novel and scalable visualization technique that uses bar charts. This interactive technique supports the ranking of items based on multiple heterogeneous attributes with different scales and semantics. It enables users to interactively combine attributes and flexibly refine parameters to explore the effect of changes in the attribute combination. This process can be employed to derive actionable insights as to which attributes of an item need to be modified in order for its rank to change. Additionally, through integration of slope graphs, LineUp can also be used to compare multiple alternative rankings on the same set of items, for example, over time or across different attribute combinations. We evaluate the effectiveness of the proposed multi-attribute visualization technique in a qualitative study. The study shows that users are able to successfully solve complex ranking tasks in a short period of time. BSD Harvard and Kitware, Inc.
Publications LineUp Web Visualization 2014-07 2014-06 LineUpWeb is the web version of the novel and scalable visualization technique. This interactive technique supports the ranking of items based on multiple heterogeneous attributes with different scales and semantics. It enables users to interactively combine attributes and flexibly refine parameters to explore the effect of changes in the attribute combination. BSD Stanford, University of Washington, Kitware, Inc. Lyra Visualization 2014-07 2014-02 Lyra is an interactive environment that makes custom visualization design accessible to a broader audience. With Lyra, designers map data to the properties of graphical marks to author expressive visualization designs without writing code. Marks can be moved, rotated and resized using handles; relatively positioned using connectors; and parameterized by data fields using property drop zones. Lyra also provides a data pipeline interface for iterative, visual specification of data transformations and layout algorithms. Visualizations created with Lyra are represented as specifications in Vega, a declarative visualization grammar that enables sharing and reuse. BSD Phronesis stat_agg Analytics 2014-07 https://github.com/kaneplusplus/stat_agg.git stats stat_agg is a Python package that provides statistical aggregators that maximize ensemble prediction accuracy by weighting individual learners in an optimal way. When used with the laputa package, learners may be distributed across a cluster of machines. The package also provides fault-tolerance when one or more learners becomes unavailable. ALv2 Phronesis flexmem Infrastructure 2014-07 https://github.com/kaneplusplus/flexmem.git stats Flexmem is a general, transparent tool for out-of-core (OOC) computing in the R programming environment. It is launched as a command line utility, taking an application as an argument. All memory allocations larger than a specified threshold are memory-mapped to a binary file. When data are not needed, they are stored on disk. It is both process- and thread-safe. ALv2 Phronesis laputa Infrastructure 2014-07 https://github.com/kaneplusplus/laputa.git stats Laputa is a Python package that provides an elastic, parallel computing foundation for the stat_agg (statistical aggregates) package. ALv2 Phronesis bigmemory Infrastructure 2014-07 http://cran.r-project.org/web/packages/bigmemory/index.html Bigmemory is an R package to create, store, access, and manipulate massive matrices. Matrices are allocated to shared memory and may use memory-mapped files. Packages biganalytics, bigtabulate, synchronicity, and bigalgebra provide advanced functionality. ALv2 Phronesis bigalgebra Infrastructure 2014-07 https://r-forge.r-project.org/scm/viewvc.php/?root=bigmemory Bigalgebra is an R package that provides arithmetic functions for R matrix and big.matrix objects. ALv2 MDA Information Systems, Inc., Jet Propulsion Laboratory, USC/Information Sciences Institute OODT Infrastructure 2014-07 https://svn.apache.org/repos/asf/oodt/ stats APACHE OODT enables transparent access to distributed resources, data discovery and query optimization, and distributed processing and virtual archives. OODT provides software architecture that enables models for information representation, solutions to knowledge capture problems, unification of technology, data, and metadata. ALv2 MDA Information Systems, Inc.,Jet Propulsion Laboratory, USC/Information Sciences Institute Wings Infrastructure 2014-07 https://github.com/varunratnakar/wings.git stats WINGS provides a semantic workflow system that assists scientists with the design of computational experiments. A unique feature of WINGS is that its workflow representations incorporate semantic constraints about datasets and workflow components, and are used to create and validate workflows and to generate metadata for new data products. WINGS submits workflows to execution frameworks such as Pegasus and OODT to run workflows at large scale in distributed resources. ALv2 MIT-LL
Publications Query By Example (Graph QuBE) Analytics 2014-07 2014-02-15 Query-by-Example (Graph QuBE) on dynamic transaction graphs. ALv2 MIT-LL
Publications Julia Analytics 2014-07 https://github.com/JuliaLang/julia.git stats Julia is a high-level, high-performance dynamic programming language for technical computing, with syntax that is familiar to users of other technical computing environments. It provides a sophisticated compiler, distributed parallel execution, numerical accuracy, and an extensive mathematical function library. MIT,GPL,LGPL,BSD MIT-LL
Publications Topic Analytics 2014-07 Pending Probabilistic Latent Semantic Analysis (pLSA) Topic Modeling. ALv2 MIT-LL
Publications SciDB Infrastructure 2014-07 https://github.com/wujiang/SciDB-mirror.git stats Scientific Database for large-scale numerical data. GPLv3 MIT-LL
Publications Information Extractor Analytics 2014-07 Pending Trainable named entity extractor (NER) and relation extractor. ALv2 Next Century Corporation Ozone Widget Framework Visualization 2014-07 https://github.com/ozoneplatform/owf.git stats Ozone Widget Framework provides a customizable open-source web application that assembles the tools you need to accomplish any task and enables those tools to communicate with each other. It is a technology-agnostic composition framework for data and visualizations in a common browser-based display and interaction environment that lowers the barrier to entry for the development of big data visualizations and enables efficient exploration of large data sets. ALv2 Next Century Corporation Neon Visualization Environment Visualization 2014-07 https://github.com/NextCenturyCorporation/neon.git stats Neon is a framework that gives a datastore agnostic way for visualizations to query data and perform simple operations on that data such as filtering, aggregation, and transforms. It is divided into two parts, neon-server and neon-client. Neon-server provides a set of RESTful web services to select a datastore and perform queries and other operations on the data. Neon-client is a javascript API that provides a way to easily integrate neon-server capabilities into a visualization, and also aids in 'widgetizing' a visualization, allowing it to be integrated into a common OWF based ecosystem. ALv2 Oculus Info Inc.
Publications ApertureJS Visualization 2014-07 https://github.com/oculusinfo/aperturejs.git stats ApertureJS is an open, adaptable and extensible JavaScript visualization framework with supporting REST services, designed to produce visualizations for analysts and decision makers in any common web browser. Aperture utilizes a novel layer based approach to visualization assembly, and a data mapping API that simplifies the process of adaptable transformation of data and analytic results into visual forms and properties. Aperture vizlets can be easily embedded with full interoperability in frameworks such as the Ozone Widget Framework (OWF). MIT Oculus Info Inc.
Publications Influent Visualization 2014-07 https://github.com/oculusinfo/influent.git stats Influent is an HTML5 tool for visually and interactively following transaction flow, rapidly revealing actors and behaviors of potential concern that might otherwise go unnoticed. Summary visualization of transactional patterns and actor characteristics, interactive link expansion and dynamic entity clustering enable Influent to operate effectively at scale with big data sources in any modern web browser. Influent has been used to explore data sets with millions of entities and hundreds of millions of transactions. MIT Oculus Info Inc.
Publications Aperture Tile-Based Visual Analytics Visualization 2014-07 https://github.com/oculusinfo/aperture-tiles.git stats New tools for raw data characterization of 'big data' are required to suggest initial hypotheses for testing. The widespread use and adoption of web-based maps has provided a familiar set of interactions for exploring abstract large data spaces. Building on these techniques, we developed tile based visual analytics that provide browser-based interactive visualization of billions of data points. MIT Oculus Info Inc.
Publications Oculus Ensemble Clustering Analytics 2014-07 https://github.com/oculusinfo/ensemble-clustering.git stats Oculus Ensemble Clustering is a flexible multi-threaded clustering library for rapidly constructing tailored clustering solutions that leverage the different semantic aspects of heterogeneous data. The library can be used on a single machine using multi-threading or distributed computing using Spark. MIT Raytheon BBN Content and Context-based Graph Analysis: PINT, Patterns in Near-Real Time Analytics 2014-07 https://github.com/plamenbbn/XDATA.git stats Patterns in Near-Real Time will take any corpus as input and quantify the strength of the query match to a SME-based process model, represent process model as a Directed Acyclic Graph (DAG), and then search and score potential matches. ALv2 Raytheon BBN Content and Context-based Graph Analysis: NILS, Network Inference of Link Strength Analytics 2014-07 https://github.com/plamenbbn/XDATA.git stats Network Inference of Link Strength will take any text corpus as input and quantify the strength of connections between any pair of entities. Link strength probabilities are computed via shortest path. ALv2 Royal Caliber
Publications GPU based Graphlab style Gather-Apply-Scatter (GAS) platform for quickly implementing and running graph algorithms Analytics 2014-07 https://github.com/RoyalCaliber/vertexAPI2.git stats Allows users to express graph algorithms as a series of Gather-Apply-Scatter (GAS) steps similar to GraphLab. Runs these vertex programs using a single or multiple GPUs - demonstrates a large speedup over GraphLab. ALv2 Scientific Systems Company, Inc., MIT, and University of Louisville BayesDB Analytics 2014-07 https://github.com/mit-probabilistic-computing-project/BayesDB.git stats BayesDB is an open-source implementation of a predictive database table. It provides predictive extensions to SQL that enable users to query the implications of their data --- predict missing entries, identify predictive relationships between columns, and examine synthetic populations --- based on a Bayesian machine learning system in the backend. ALv2 Scientific Systems Company, Inc., MIT, and University of Louisville Crosscat Analytics 2014-07 https://github.com/mit-probabilistic-computing-project/crosscat.git stats CrossCat is a domain-general, Bayesian method for analyzing high-dimensional data tables. CrossCat estimates the full joint distribution over the variables in the table from the data via approximate inference in a hierarchical, nonparametric Bayesian model, and provides efficient samplers for every conditional distribution. CrossCat combines strengths of nonparametric mixture modeling and Bayesian network structure learning: it can model any joint distribution given enough data by positing latent variables, but also discovers independencies between the observable variables. ALv2 Sotera Defense Solutions, Inc.
Publications Zephyr Infrastructure 2014-07 http://github.com/Sotera/zephyr stats Zephyr is a big data, platform agnostic ETL API, with Hadoop MapReduce, Storm, and other big data bindings. ALv2 Sotera Defense Solutions, Inc.
Publications Page Rank Analytics 2014-07 https://github.com/Sotera/page-rank.git stats Sotera Page Rank is a Giraph/Hadoop implementation of a distributed version of the Page Rank algorithm. ALv2 Sotera Defense Solutions, Inc.
Publications Louvain Modularity Analytics 2014-07 https://github.com/Sotera/distributed-louvain-modularity.git stats Giraph/Hadoop implementation of a distributed version of the Louvain community detection algorithm. ALv2 Sotera Defense Solutions, Inc.
Publications Spark MicroPath Analytics 2014-07 https://github.com/Sotera/aggregate-micro-paths.git The Spark implementation of the micropath analytic. ALv2 Sotera Defense Solutions, Inc.
Publications ARIMA Analytics 2014-07 https://github.com/Sotera/rhipe-arima stats Hive and RHIPE implementation of an ARIMA analytic. ALv2 Sotera Defense Solutions, Inc.
Publications Leaf Compression Analytics 2014-07 https://github.com/Sotera/leaf-compression.git stats Recursive algorithm to remove nodes from a network where degree centrality is 1. ALv2 Sotera Defense Solutions, Inc.
Publications Correlation Approximation Analytics 2014-07 https://github.com/Sotera/correlation-approximation stats Spark implementation of an algorithm to find highly correlated vectors using an approximation algorithm. ALv2 Stanford University - Boyd
Publications QCML (Quadratic Cone Modeling Language) Analytics 2014-07 https://github.com/cvxgrp/qcml.git stats Seamless transition from prototyping to code generation. Enable ease and expressiveness of convex optimization across scales with little change in code. ALv2 Stanford University - Boyd
Publications PDOS (Primal-dual operator splitting) Analytics 2014-07 https://github.com/cvxgrp/pdos.git stats Concise algorithm for solving convex problems; solves problems passed from QCML. ALv2 Stanford University - Boyd
Publications SCS (Self-dual Cone Solver) Analytics 2014-07 https://github.com/cvxgrp/scs.git stats Implementation of a solver for general cone programs, including linear, second-order, semidefinite and exponential cones, based on an operator splitting method applied to a self-dual homogeneous embedding. The method and software supports both direct factorization, with factorization caching, and an indirect method, that requires only the operator associated with the problem data and its adjoint. The implementation includes interfaces to CVX, CVXPY, matlab, as well as test routines. This code is described in detail in an associated paper, at http://www.stanford.edu/~boyd/papers/pdos.html (which also links to the code). ALv2 Stanford University - Boyd
Publications ECOS: An SOCP Solver for Embedded Systems Analytics 2014-07 https://github.com/ifa-ethz/ecos.git stats ECOS is a lightweight primal-dual homogeneous interior-point solver for SOCPs, for use in embedded systems as well as a base solver for use in large scale distributed solvers. It is described in the paper at http://www.stanford.edu/~boyd/papers/ecos.html. ALv2 Stanford University - Boyd
Publications Proximal Operators Analytics 2014-07 https://github.com/cvxgrp/proximal.git stats This library contains sample implementations of various proximal operators in Matlab. These implementations are intended to be pedagogical, not the most performant. This code is associated with the paper Proximal Algorithms by Neal Parikh and Stephen Boyd. ALv2 Stanford University - Hanrahan
Publications imMens Visualization 2014-07 https://github.com/StanfordHCI/imMens.git stats imMens is a web-based system for interactive visualization of large databases. imMens uses binned aggregation to produce summary visualizations that avoid the shortcomings of standard sampling-based approaches. Through data decomposition methods (to limit data transfer) and GPU computation via WebGL (for parallel query processing), imMens enables real-time (50fps) visual querying of billion+ element databases. BSD Stanford University - Hanrahan
Publications trelliscope Visualization 2014-07 https://github.com/hafen/trelliscope.git stats Trellis Display, developed in the 90s, also divides the data. A visualization method is applied to each subset and shown on one panel of a multi-panel trellis display. This framework is a very powerful mechanism for all data, large and small. Trelliscope, a layer that uses datadr, extends Trellis to large complex data. An interactive viewer is available for viewing subsets of very large displays, and the software provides the capability to sample subsets of panels from rigorous sampling plans. Sampling is often necessary because in most applications, there are too many subsets to look at them all. BSD Stanford University - Hanrahan
Publications RHIPE: R and Hadoop Integrated Programming Environment Infrastructure 2014-07 https://github.com/saptarshiguha/RHIPE.git stats In Divide and Recombine (D&R), big data are divided into subsets in one or more ways, forming divisions. Analytic methods, numeric-categorical methods of machine learning and statistics plus visualization methods, are applied to each of the subsets of a division. Then the subset outputs for each method are recombined. D&R methods of division and recombination seek to make the statistical accuracy of recombinations as large as possible, ideally close to that of the hypothetical direct, all-data application of the methods. The D&R computational environment starts with RHIPE, a merger of R and Hadoop. RHIPE allows an analyst to carry out D&R analysis of big data wholly from within R, and use any of the thousands of methods available in R. RHIPE communicates with Hadoop to carry out the big, parallel computations. ALv2 Stanford University - Hanrahan
Publications Riposte Analytics 2014-07 https://github.com/jtalbot/riposte.git stats Riposte is a fast interpreter and JIT for R. The Riposte VM has 2 cooperative subVMs for R scripting (like Java) and for R vector computation (like APL). Our scripting code has been 2-4x faster in Riposte than in R's recent bytecode interpreter. Vector-heavy code is 5-10x faster. Speeding up R can greatly increases the analyst's efficiency. BSD Stanford University - Olukotun
Publications Delite Infrastructure 2014-07 https://github.com/stanford-ppl/Delite.git stats Delite is a compiler framework and runtime for parallel embedded domain-specific languages (DSLs). BSD Stanford University - Olukotun
Publications SNAP Infrastructure 2014-07 https://github.com/snap-stanford/snap stats Stanford Network Analysis Platform (SNAP) is a general purpose network analysis and graph mining library. It is written in C++ and easily scales to massive networks with hundreds of millions of nodes, and billions of edges. It efficiently manipulates large graphs, calculates structural properties, generates regular and random graphs, and supports attributes on nodes and edges. BSD SYSTAP, LLC bigdata Infrastructure 2014-07 https://bigdata.svn.sourceforge.net/svnroot/bigdata/ stats Bigdata enables massively parallel graph processing on GPUs and many core CPUs. The approach is based on the decomposition of a graph algorithm as a vertex program. The initial implementation supports an API based on the GraphLab 2.1 Gather Apply Scatter (GAS) API. Execution is available on GPUs, Intel Xenon Phi (aka MIC), and multi-core GPUs. GPLv2 SYSTAP, LLC mpgraph Analytics 2014-07 http://svn.code.sf.net/p/mpgraph/code/ stats Mpgraph enables massively parallel graph processing on GPUs and many core CPUs. The approach is based on the decomposition of a graph algorithm as a vertex program. The initial implementation supports an API based on the GraphLab 2.1 Gather Apply Scatter (GAS) API. Execution is available on GPUs, Intel Xenon Phi (aka MIC), and multi-core GPUs. ALv2 UC Davis Gunrock Analytics 2014-07 https://github.com/gunrock/gunrock.git stats Gunrock is a CUDA library for graph primitives that refactors, integrates, and generalizes best-of-class GPU implementations of breadth-first search, connected components, and betweenness centrality into a unified code base useful for future development of high-performance GPU graph primitives. ALv2 Draper Laboratory
Publications Analytic Activity Logger Infrastructure 2014-07 https://github.com/draperlab/xdatalogger.git stats Analytic Activity Logger is an API that creates a common message passing interface to allow heterogeneous software components to communicate with an activity logging engine. Recording a user's analytic activities enables estimation of operational context and workflow. Combined with psychophysiology sensing, analytic activity logging further enables estimation of the user's arousal, cognitive load, and engagement with the tool. ALv2 University of California, Berkeley
Publications BDAS Infrastructure 2014-07 N/A BDAS, the Berkeley Data Analytics Stack, is an open source software stack that integrates software components being built by the AMPLab to make sense of Big Data. ALv2, BSD University of California, Berkeley
Publications Spark Infrastructure 2014-07 https://github.com/mesos/spark.git stats Apache Spark is an open source cluster computing system that aims to make data analytics both fast to run and fast to write. To run programs faster, Spark offers a general execution model that can optimize arbitrary operator graphs, and supports in-memory computing, which lets it query data faster than disk-based engines like Hadoop. To make programming faster, Spark provides clean, concise APIs in Python, Scala and Java. You can also use Spark interactively from the Scala and Python shells to rapidly query big datasets. ALv2 University of California, Berkeley
Publications Shark Infrastructure 2014-07 https://github.com/amplab/shark.git stats Shark is a large-scale data warehouse system for Spark that is designed to be compatible with Apache Hive. It can execute Hive QL queries up to 100 times faster than Hive without any modification to the existing data or queries. Shark supports Hive's query language, metastore, serialization formats, and user-defined functions, providing seamless integration with existing Hive deployments and a familiar, more powerful option for new ones. ALv2 University of California, Berkeley
Publications BlinkDB Infrastructure 2014-07 https://github.com/sameeragarwal/blinkdb.git stats BlinkDB is a massively parallel, approximate query engine for running interactive SQL queries on large volumes of data. It allows users to trade-off query accuracy for response time, enabling interactive queries over massive data by running queries on data samples and presenting results annotated with meaningful error bars. To achieve this, BlinkDB uses two key ideas: (1) An adaptive optimization framework that builds and maintains a set of multi-dimensional samples from original data over time, and (2) A dynamic sample selection strategy that selects an appropriately sized sample based on a query's accuracy and/or response time requirements. We have evaluated BlinkDB on the well-known TPC-H benchmarks, a real-world analytic workload derived from Conviva Inc. and are in the process of deploying it at Facebook Inc. ALv2 University of California, Berkeley
Publications Mesos Infrastructure 2014-07 https://git-wip-us.apache.org/repos/asf/mesos.git stats Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, MPI, Hypertable, Spark, and other applications on a dynamically shared pool of nodes. ALv2 University of California, Berkeley
Publications Tachyon Infrastructure 2014-07 https://github.com/amplab/tachyon.git stats Tachyon is a fault tolerant distributed file system enabling reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce. It achieves high performance by leveraging lineage information and using memory aggressively. Tachyon caches working set files in memory, and enables different jobs/queries and frameworks to access cached files at memory speed. Thus, Tachyon avoids going to disk to load datasets that are frequently read. BSD University of Southern California
Publications goffish Infrastructure 2014-07 https://github.com/usc-cloud/goffish.git stats The GoFFish project offers a distributed framework for storing timeseries graphs and composing graph analytics. It takes a clean-slate approach that leverages best practices and patterns from scalable data analytics such as Hadoop, HDFS, Hive, and Giraph, but with an emphasis on performing native analytics on graph (rather than tuple) data structures. This offers an more intuitive storage, access and programming model for graph datasets while also ensuring performance optimized for efficient analysis over large graphs (millions-billions of vertices) and many instances of them (thousands-millions of graph instances). ALv2

Original post on Hacker News

Stephen Law: How the US Treasury imposes sanctions on me and every other "Stephen Law" on the planet - my letter to OFAC

Comments: "Stephen Law: How the US Treasury imposes sanctions on me and every other "Stephen Law" on the planet - my letter to OFAC"

URL: http://stephenlaw.blogspot.com/2014/02/how-us-treasury-imposes-sanctions-on-me.html


Right, here's another thing I am getting off my chest - email letter to OFAC (edited slightly from version sent).

Dear OFAC

This correspondence is copied to my UK Member of Parliament The Right Hon. Andrew Smith. Please copy him into your reply.

My name is "Stephen Law". The name "Stephen Law" appear on OFAC's "specially designated nationals" list:

Here is the actual OFAC listing for "Stephen Law", alias of "Steven Law"

https://ofac.data-list-search.com/Entities/ByName/steven-law

This person is Burmese and is suspected by US Treasury of drug trafficking. He is the son of Lo Hsing Han (dubbed by US Treasury as "The Godfather of Heroin") and has a Singaporean wife. His addresses, as listed by you, are all in Burma and Singapore. None are in the UK.

I have discovered that, as a result of this listing, US Customs block shipments of goods to me here in the UK. Also when people try to wire me money from abroad (not just from the US, but from anywhere), for e.g. occasional travel expenses for academic conference attendance, the payment is interrupted and various checks are made before the funds are released. This became so bad during one period (a series of payments every single one of which triggered a block) that I had to switch to a different bank account. At no point was I told why this was happening (i.e. that you, OFAC, are responsible). The banks concerned believe they must keep this information from me (I was told this by my bank branch). Hence it took me many months to figure out what the source of the problem was: OFAC/US Treasury.

It appears any "Stephen Law" anywhere in the world will suffer this same treatment, as indeed will anyone who merely happens to have the same name or alias as one of your "specially designated nationals". This has proved frustrating, time-consuming and also costly to me personally. E.g. I have  paid US$77 postage for goods it turns out I can never receive because they are returned by US customs to the US vendor because my name is listed. As a result of the OFAC listing, I cannot now order goods from - or receive gifts from friends and relatives in - the United States.

Can you inform me: given I am very obviously NOT the Burmese Stephen Law:

(i) how I can avoid having all goods shipped to me from the US to my UK address being blocked and returned to sender by US customs?

(ii) how I can avoid my own bank repeatedly asking me who I am (and requesting information including my DOB, which they already possess) before unblocking any payment from abroad?

My bank knows who I am, and they know I am not the Burmese "Stephen Law" on the specially designated nationals list, but still I have to go through this same rigmarole every single time money is wired to me. How do I avoid this please?

Yours faithfully

Stephen Law

PS Ofac-caused delays to payments to me can run into weeks. On one occasion I ran up overdraft charges as a result of not receiving funds blocked by OFAC.

PPS I was interviewed by Foreign Policy magazine about all this a short while ago.Also interviewed by News Hour on BBC World Service.

Original post on Hacker News

The Science Behind 'Brain Training' - Dan Hurley - The Atlantic

Comments: "The Science Behind 'Brain Training' - Dan Hurley - The Atlantic"

URL: http://www.theatlantic.com/health/archive/2014/02/the-science-behind-brain-training/283634/


Increasing fluid intelligence has proven beneficial for people diagnosed with ADHD, and selling memory improvement is a big business. Are the claims overheated? 

anvilon/flickr

In 2002, Torkel Klingberg, a psychologist at Sweden’s Karolinska Institute, published a study involving 14 children with attention deficit hyperactivity disorder. All of the children were asked to spend a total of 10.5 hours, over five weeks, practicing computerized games that put demands on their working memory—their moment-by-moment attention and ability to juggle and analyze the objects of their attention.

Seven of the children played the games only at the beginner’s level; for the other seven, the games became progressively harder as the children got better. At the study’s end, the group who trained progressively not only improved on the games, but also on other measures of working memory. Their hyperactivity, as measured by head movement, lessened. And incredibly, even bizarrely by the standards of orthodoxy then holding sway, they also did much better on the Raven’s progressive matrices, long regarded as psychology’s single best measure of fluid intelligence. If the results were to be believed, the kids had gotten smarter.

"We don't have these huge studies that drug companies have. On the other hand, this is not something that is dangerous."

At first scoffed at, the little study has since led to dozens of other studies (15 of them listed here) aimed at replicating and expanding its finding. It also quickly led Klingberg and the Karolinska Institute to form a company, Cogmed, to turn working-memory training into a business. The initial target market was children with ADHD, whose parents hoped to find something other than drugs to improve their children’s attention; but soon the market expanded to treatments for both adults and children with a variety of cognitive disorders. By 2010, in a step suggesting just how vast a business this brain training could be, Cogmed was sold to Pearson, the largest education company in the world.

“Cogmed is a leader in the emerging field of evidence-based cognitive training,” the company states on its website. “We have scientifically validated research showing that Cogmed training provides substantial and lasting improvements in attention for people with poor working memory—in all age groups. That makes Cogmed’s products the best-validated products on the market.”

Aren’t claims like this rather overheated? I posed that question to Klingberg at Joe Coffee, a tiny, crowded coffee shop on West 23rd Street in Manhattan, where he was in town to give a talk at Columbia University. He wore a black leather jacket and a momentary scowl, having heard such critical questions many times before.

“Yeah, well,” he said with a shrug. “We did start to do research in 1999. Of course you can say that we still don’t have—we should wait another 10 years until we have thousands of participants. This is a general problem with cognitive training studies that we don’t have these huge studies that drug companies have. On the other hand, this is not something that is dangerous.”

He pointed out that Cogmed claims only that it can improve working memory, not fluid intelligence per se—even though many studies have found that working memory and fluid intelligence are closely related.

“What we see, over and over again,” he said, “is improvement of working memory and also of attention, including attention in everyday life. This is not everything, but it’s good enough for me if we can have that. Working-memory problems and attention problems are huge for many children and adults. Right now I don’t have any financial interest in Cogmed anymore. The influence I have had over Cogmed has been to make them very cautious. They don’t make claims about rejuvenating your brain or improving intelligence.”

At a total cost to families of about $2,000 for the 25 sessions, Cogmed compares favorably with many other kinds of ADHD treatments.

I asked him to describe exactly what kind of computerized training tasks Cogmed offers.

“There are 12 tasks,” he said. “They’re all visuospatial. The role of attention in working memory almost always has a spatial dimension. When you’re paying attention—even when you’re paying attention to me talking here in this café—there’s a spatial component. When there’s a loud noise, you might shift your attention to where it’s coming from. Being able to maintain your spatial focus on me is important for you right now. Even though it’s words coming from me, there’s an important component of space. So if you can improve the stability of that spatial aspect, you will be better at visuospatial tasks and be better at keeping your focus on me rather than on that noise over there.”

Still wanting a better sense of the exercises Cogmed offers, I scheduled a meeting with a clinical psychologist, Nicole Garcia, who offers the training just a few miles from my home in Montclair, New Jersey. She allowed me to sit down at her computer to play a handful of the games. (She emphasized that Cogmed calls them “training tasks,” not games. But they looked like games to me.)

I clicked one called Hidden, which showed a standard numeric keypad, the kind used on cell phones and calculators. The keypad was hidden while a man’s voice recited a short list of numbers. When his list was complete, the keypad reappeared, and I was supposed to click on the list—in reverse order. Another game showed a circle with nine smaller circles strung along it like carriages on a Ferris wheel. As the big circle slowly rotated in a clockwise direction, the little circles lit up in a random sequence. Once the sequence was completed, I had to click on the little circles in the same order.

All of the games were easy on the first pass, but immediately grew hard enough—meaning that the sequence to be remembered grew longer and was presented faster—that I began making mistakes.

Having offered Cogmed to a few dozen of her patients, as young as six and as old as 63 (including a successful attorney whose ADHD was diagnosed in her forties), Garcia said she’s convinced of its benefits, sometimes in combination with medication and sometimes on its own. And at a total cost to families of about $2,000 for the 25 sessions, she said, it compares favorably with many other kinds of ADHD treatments.

Where Cogmed beats all other forms of cognitive training is in the number of published, randomized clinical trials demonstrating its benefits and the number of trials still under way, led by independent researchers at leading institutions without any commercial connection to the company.

Julie Schweitzer, director of the ADHD Program at the University of California in Davis’s MIND Institute, conducted a randomized study of children diagnosed with ADHD.  When published in July 2012 in the journal Neurotherapeutics, Schweitzer’s study found that children in the placebo group spent just as much time off-task at the end of the study as they had at the beginning, but those who trained on Cogmed sharply increased the amount of time they spent doing school work.

"Around 20 to 40 percent of children treated for leukemia will end up with cognitive changes. For those treated for brain tumors, the figure is around 60 to 80 percent."

Children who have survived cancer are another group often in need of cognitive rehabilitation. “Somewhere around 20 to 40 percent of children treated for leukemia will end up with cognitive changes over time,” said Kristina K. Hardy, a neuropsychologist at Children’s National Medical Center in Washington, D.C. “For those treated for brain tumors, the figure is conservatively around 60 to 80 percent.”

What distinguishes these young survivors from most others seeking cognitive rehabilitation is that the effects of radiation or chemotherapy on the brain become apparent only with the passage of time. Immediately following treatment, a recent study found, survivors of acute lymphoblastic leukemia showed no significant change in their verbal IQ scores, but by early adulthood, their scores had dropped by an average of 10.3 points. 

In 2012, Hardy reported the results of a pilot study comparing Cogmed to a placebo form of computerized. Among 20 children who had survived either brain cancer or leukemia, those who trained with Cogmed saw substantial improvements compared to the placebo group on their visual working memory and in parent-rated learning problems.

Most recently, children with Down syndrome have been shown to benefit from Cogmed. “Following training,” concluded a study published last June, “performance on trained and untrained visuospatial short-term memory tasks was significantly enhanced for children in the intervention group. This improvement was sustained four months later. These results suggest that computerized visuospatial memory training in a school setting is both feasible and effective for children with Down syndrome.”

Brian Skotko, co-director of the Down Syndrome Program at Massachusetts General Hospital, told me, “If Cogmed was a drug, everyone would call this study groundbreaking.”

Not all studies of Cogmed have been positive. A large one published in October found little benefit. But as Klingberg has written in defense of Cogmed in particular and working-memory training in general: “Working memory training is still a young field of research. As with all science, no single experiment explains everything, and results are never perfectly consistent … Many questions remain. But there is no going back to the notion that working memory capacity is fixed.”

This post is adapted from Dan Hurley's Smarter: The New Science of Building Brain Power.

Original post on Hacker News

Explore – …the appointee’s wife was granted a divorce from...

Comments: "...was granted a divorce because he was constantly working calculus problems"

URL: http://explore.noodle.org/post/73954140444/the-appointees-wife-was-granted-a-divorce-from


Original post on Hacker News

Phaser

Comments: "Phaser"

URL: http://phaserapp.com


Original post on Hacker News

First geologic map of Ganymede made with Voyager data | Ars Technica

Comments: "First geologic map of Ganymede made with Voyager data | Ars Technica"

URL: http://arstechnica.com/science/2014/02/first-geologic-map-of-ganymede-made-with-voyager-data/


Imagery of Ganymede's surface (right) and the new map of its geology (left). USGS

Maps have always been an integral part of exploration. They take the in out of terra incognita. Some things are easier to map than others, of course. The geology of a world a few hundred million miles away is one of those other things. Nevertheless, the United States Geological Survey just released a geologic map of Jupiter’s moon Ganymede—an icy satellite larger than Mercury.

The map was created through the hard work of a team led by Wheaton College’s Geoffrey Collins using imagery from the Voyager probes and the more recent Galileo mission. Much in the way that geologists can determine the relative ages of Earth rocks by noting which rocks cut into or through others, Ganymede’s surface can tell us about its own geologic history.

The researchers identified three basic periods in that history, which they named the Gilgameshan, Harpagian, and Nicholsonian periods. The oldest is marked by an abundance of impact craters, the second by extensive tectonic activity that altered and deformed the surface, and the youngest by an absence of significant activity.

A PDF of the detailed map, which might look great on your wall, is available on the USGS website.

Original post on Hacker News

UpCounsel Launches Outside General Counsel Program For Companies Needing Long-Term Legal Help | TechCrunch

Comments: "UpCounsel Launches Outside General Counsel Program For Companies Needing Long-Term Legal Help | TechCrunch"

URL: http://techcrunch.com/2014/02/14/upcounsel-outside-general-counsel/


Attorney marketplace UpCounsel has spent the last several months helping startups and other small businesses get affordable legal help. But for the most part, that help has mainly been focused on short-term projects that don’t require a ton of assistance over a longer period of time.

The startup hopes to change that with a new service that will connect technology companies in San Francisco with an outside general counsel to replace the legal help they’d usually get from a traditional law firm. Those firms can charge up to $800 per hour to work with a partner, but UpCounsel believes that by setting companies up with attorneys on its platform, it can drastically reduce that cost over time.

The Outside General Counsels it connects startups with are former senior associates and partners from large firms who have previously served in the general counsel role of technology companies. The only difference is now they’re working virtually through the UpCounsel platform.

To make sure that the general counsels are a good fit, UpCounsel does interviews with interested companies to determine what their needs are, and then tries to pair them with attorneys who understand their business and have the correct skill sets to support them. Attorneys get a company profile and dossier to review only if they are selected by the company to possibly represent them.

More than just making the connection between startups and the outside general counsel, UpCounsel also handles all admin and support for them. That includes billing, but also means helping them to find paralegals as well.

As part of the program, UpCounsel is also opening up not just to attorneys, but to professionals who have served as paralegals to support them. Again, since it doesn’t have all the overhead of the big firms, paralegals can be billed at about a third of what the big firms charge for their hourly work.

In addition to the lower cost, UpCounsel believes that its outside general counsels will be more responsive to legal requests than the folks who work at more traditional firms that are loaded up on casework. CEO Matt Faustman tells me that in its early trial, some startups have used the program to complement their existing firms for when they need more immediate help.

But the platform is also seeing some startups move completely to adopt its outside general counsels, with about 10 totally jumping ship from some big firms you’ve probably heard of.

Now that it’s launched, we’ll see how well the program actually works. In the meantime, UpCounsel has raised $1.5 million in seed funding from folks that include Homebrew, Bobby Yazdani, SV Angel, Collaborative Fund, Haroon Mokhtarzada, and other angels.

Original post on Hacker News

Rendered Prose Diffs · GitHub

Comments: "Rendered Prose Diffs · GitHub"

URL: https://github.com/blog/1784-rendered-prose-diffs


Today we are making it easier to review and collaborate on prose documents. Commits and pull requests including prose files now feature source and rendered views.

Click the "rendered" button to see the changes as they'll appear in the rendered document. Rendered prose view is handy when you're adding, removing, and editing text:

Or working with more complex structures like tables:

Non-text changes appear with a low-key dotted underline. Hover over the text to see what has changed:

Building great software is about more than code. Whether you're writing docs, planning development, or blogging what you've learned, better prose makes for better products. Go forth and write together!

Need help or found a bug? Contact us.

Original post on Hacker News

Real-time tech support with Olark — break the bit

Comments: "Real-time tech support with Olark integration into your client-side app"

URL: http://breakthebit.org/post/75335427347/real-time-tech-support-with-olark


Foreword

We came a long way since we started the public beta testing of Dubjoy, the video voice-over solution in the browser.

We’re targeting a pretty specific public: Language Service Providers, voice actors, voice talents, translators and interpreters. This is mostly a non-technical blend, that like most people, can’t help you efficiently debug, reproduce or describe problems as they occur.

We came a long way since then, with most of the problems being solved and the software approaching production quality fast.

But still, we were desperate for a way to better assist our customers, preferably in real-time.

There are many moving parts to a voice recording app in the browser. And moving parts always make for a fun array of problems customers encounter.

We have to be able to assist people while learning how to use the software for the first time. There can be workflow issues, microphone sensitivity and permission problems, Flash versions, etc.

Communication with your customers is key, and I can’t describe how overwhelmingly positive the reactions have been.

They love to see that you have their back at all times.

Required capabilities

So our ideal system would have the following capabilities:

  • real-time feedback and help,
  • log forwarding,
  • error forwarding,
  • diagnostic checks,
  • recovery routines,
  • reset routines,
  • system information.

And we set out to find something out there that could help.

Olark

If you’re selling anything on the web and don’t use Olark you’re missing out.

They’re making a chat widget, that you integrate into your site. Once installed your website visitors appear as chat contacts in your IM program like Google Talk.

You can click on anyone and start talking them up, or wait that someone need help and clicks the “Chat with us” button.

So this tool by itself eliminates our first capability requirement: real-time feedback and help.

But as we delved deeper into their developer API documentation we discovered that you can easily extend the functionality of the chat widget by creating your own custom commands and binding to certain events, like when the conversation begins.

Anatomy of our Olark integration

Olark has a nice feature that chat operators can issue commands.

A command is simply any word prefixed with an exclamation, like !debugon.

The “brain” of the integration is a simple parser, that hooks up to the api.chat.onCommandFromOperator event.

A simple example of this, an implementation of the !explainer command:

olark 'api.chat.onCommandFromOperator', (event) ->
 # !explainer
 # Show the popup with the explainer video.
 if event.command.name is 'explainer'
 V.olark.send "[EXPLAINER] Showing popup"
 V.overlay.help()

So Dubjoy’s current integration looks like this (you can list the current commands by issuing !dubjoy:

# !dubjoy
# Show Dubjoy help and commands.
if event.command.name is 'dubjoy'
 V.olark.send "[HELP] Dubjoy Olark integration\n"
 V.olark.send "!status - Status report (env, kombajn, video, frags, app)"
 V.olark.send "!debugon [<level>] - Turn on log fwd at a level"
 V.olark.send "!debugoff - Turn off log fwd"
 V.olark.send "!reload [<restore>] - Force reload and restore state"
 V.olark.send "!undo - Undo last step"
 V.olark.send "!explainer - Show the explainer popup"
 V.olark.send "!fragments - Output detailed fragments data"
 V.olark.send "!fragmiss - Output fragments that are missing audio"
 V.olark.send "!fragclear <fragment_id> - Clear audio data for fragment"
 V.olark.send "!fragseek <fragment_id> - Seek to fragment"
 V.olark.send "!diagnose - Run system checks"
 V.olark.send "!system <details> - Show system information"
 V.olark.send "!video - Show video information, such as YouTube ID, etc."
 V.olark.send "!mic - Show if mic access is allowed"
 V.olark.send "!audio - Show the average and latest audio levels"
 V.olark.send "!audiotest - Send the audio test URL and instructions"
 V.olark.send "!exec <command> - Execute command in V. namespace (e.g.: '!exec video.play()' would execute 'V.video.play()'"

We have a variety of commands for performing basic to advanced tasks to help the customer, run diagnostic tests and check system information.

If you prefer reading code to prose and to learn more, check out our Olark integration class.

Real-time feedback and help

It’s super useful to be able to help customers who are lost or just in the app for the first time.

One of our options is to just issue a !explainer command and this pops up a short video on the customer’s screen, showing her how to use the app.

Using Olark this way makes customers feel like you always have their back. And they appreciate this and in turn easily forgive you the early bugs.

Log forwarding and error reporting

We rely heavily on logging to know in-depth what’s going on in our application. Everything from events firing, video currentTime updating to audio encoding progress is being logged, with various levels of specificity.

It doesn’t hurt to be instantly alerted whenever a breaking error occurs on the client side either. It instantly pops up a chat window to all of our engineers and the one that responds to it first, takes care of the customer.

Now imagine this scenario:

A customer is trying to voice-over a video in Dubjoy and gets stuck with the voice not recording.

She finds the Olark widget and writes us “Help! My voice is not being recorded!”.

We reply “Just a second” and turn on log forwarding with a !debugon custom command implemented for the Olark API.

This starts to forward all of the logging data the customer is generating instead of in his console to your chat window. All in real-time.

This is beyond powerful, you can instantly see what the customer is doing or clicking and of course what the app itself is doing.

How do we do it?

We have some central logging routines that we use throughout the app.

This made it trivial to plug in some code, that also sends this through Olark. This isn’t done by default, but rather the current operator can turn this on or off as simple as issuing the !debugon command.

This is a snippet from the logging routine that integrates with Olark:

# Log to Olark
if V.olark 
 if output_routine is 'error'
 # Log errors ALWAYS to Olark.
 V.olark.log type: 'ERROR', text: msg, args: args
 else if V.olark.debug and V.olark.debug is true
 # Forward logs to Olark if debugging is turned on.
 V.olark.log text: msg, args: args

A typical use case looks like this:

Diagnostic checks

It’s useful to have a single command that runs some basic diagnostic checks on the customer’s system.

We quickly check if the microphone permissions are set correctly, if the mic sensitivity is in order, Flash version, native MP4 support, video buffer status and more.

Here’s a whipped !diagnose command in action:

We also have detailed audio diagnosis commands like !audio that show the gain levels of the last recording the customer did. This way we can quickly see if the mic’s not getting any information through or the sensitivity is too low, etc.

Recovery and reset routines

It’s useful that if a customers’ session gets corrupted somehow we can trigger a reload on his end and choose whether we want to restore his data or start with a clean slate.

System information

Olark by itself shows some customer’s system information, like browser version in the tooltip of your IM app.

But it’s useful for us to have a manual way of bringing up some relevant information.

By issuing !system we get a bunch of useful information about his browser and system.

By issuing !video we get crucial information about the video he’s currently working on.

Conclusion

We’re extremely happy with the way this turned out and so are our customers.

The extensibility of the Olark platform offers unlimited potential for customer support in any JavaScript client-side application.

Original post on Hacker News

Cryptic Crossword: Amateur Crypto and Reverse Engineering

Comments: "Cryptic Crossword: Amateur Crypto and Reverse Engineering"

URL: http://www.muppetlabs.com/~breadbox/txt/acre.html


Original post on Hacker News

Making Mistakes - Aaron's Blog

Comments: " Making Mistakes - Aaron's Blog "

URL: http://www.aaronkharris.com/making-mistakes


At 10 years old, I lost a trillion dollar bet to my older brother. I bet him it was Wednesday. It was actually Thursday. I was very sure of myself (for reasons I can't recall), and saw a good opportunity to shave off some of the debt I owed him. Luckily, he has yet to call the debt. At the time, all I felt was keen embarrassment at my stupidity. As time went on It became clear I had learned a valuable lesson about taking deals that seem too good to be true.

Though mistakes that I make can be painful in the near term (and sometimes in the long term), I've found that they're a critical part of how I learn. I'm fairly certain that the mistakes I made which led to the end of Tutorspree taught me far more about how startups work than immediate success would have.

Success feels good

Success is usually idiosyncratic. Strategies that lead to success are self-validating. We build narratives around the cause and effect of success based on incomplete information and our own biases [1]. This makes it very easy to falsely attribute success to the factors that are easily seen without doing the work necessary to properly understand what happened. This tendency gets even stronger when we're looking at our own successes. At that point, ego starts to discount things like luck, which are frequently a huge part of success. Because success makes us feel good, it lulls our faculties for critical thinking, which almost by definition means it is harder to learn deep lessons [2].

Making mistakes doesn't give us the same kind of happy feelings that success does. Because the mistakes hurt, we investigate them more closely. Because we're thinking critically and comparing causal chains leading up to and flowing from mistakes, we're more likely to consider complexity and examine why things really happened as they did. That doesn't necessarily mean that each time I make a mistake I learn a critically important lesson about myself, but by thinking hard enough, I generally do learn something useful [3].

Mistakes vs. Failure

Near as I can tell, I'm not alone in believing that mistakes are a valuable tool for learning. In fact, in Silicon Valley, there's a tendency to talk about failure as a point of pride. But there's a disconnect between how we talk about Failure (intentional big "F") and how we talk about mistakes. While the internet is littered with post-mortems on failed companies, it's rare to find founders or investors who will freely admit to being wrong about a public comment or investment [4].

That doesn't make sense given what we know about mistakes. It does, however, start to make more sense with the addition of two other factors. The first is the increasing permanence and public nature of all media. It's relatively easy for me to admit mistakes to small and trusted groups because I don't fear malicious repercussions.[5] As that circle expands, the difficulty of admitting the mistake increases because I don't trust the intentions or actions of everyone in it. In the competition between wanting to learn by admitting mistakes and wanting to not be perceived as stupid or attacked for the same, not looking stupid wins.

This directly informs the second factor: increasing obsession with "personal brand." Personal brand isn't a new concept, but it has become increasingly important to more people because of the pressure to constantly tell the story of your life in public through social media. That publicity + the expanded circle again leads back to wanting to present a perfect image. Through that lens, each instant broadcast that is inconsistent with that narrative and "off-brand" appears to be a public failure in front of an untrusted circle. Not only do we look stupid for making a mistake; the mistake jeopardizes the narrative we've built about ourselves. [6]

Even if "building a brand" isn't something we consciously think about, knowing how public everything is makes being honest about mistakes hard. So how do we get as comfortable looking at recent mistakes as we are looking at the ones in the distant past? We probably can't - there's too much baggage. But I think there are ways to start moving in the right direction. Recognizing the cycle that makes it hard to admit mistakes is a good start. Developing a close friend, set of friends, or mentor, with whom you can speak honestly is another step. Most importantly, I think we probably need to take it easy and remember that no mistake, public or private, is likely to be the defining moment of our lives [7].  At the same time, cutting some slack for others who make mistakes will likely do a lot to ease the culture of recrimination/fear that has built up around making mistakes.[8] Really, we just need to be decent and thoughtful. Hard, but important.

__

[1] Fitting facts into a story is a common trap that seems to go the base of how our brains are built.

[2] If you happen to be watching the success of someone you dislike, the reverse might be true. Your critical thinking faculties might be working just fine, though your conclusions and avenues of investigation might get clouded by envy or jealousy.

[3] That might be "be careful when stopping with your new clipless pedals." Maybe small, but better to learn the lesson than assume it was easy.

[4] Interestingly, you'll see more evidence of VCs admitting to investments they missed than investments they should not have made. See Bessemer's Anti-Portfolio

[5] That certainly wasn't always easy. I've had to work on being able to admit my mistakes. The benefits I've reaped by doing so have made it easier and easier.

[6] Truth is, I haven't prioritized my personal brand that much. I firmly believe that trying to build a brand is the best way to get the reputation of just being a scenester. I think I should be judged on the things I actually do with and for the people around me. That, however, is a completely different topic.

[7] With the possible exception of Bill Buckner in and around Boston.

[8] That may mean you lose the chance to show Twitter how clever and snarky you can be. That's a good trade in the long run.

Original post on Hacker News

The Coinbase Blog — Introducing “BitHack”: Hackathon by Coinbase

Comments: "The Coinbase Blog — Introducing “BitHack”: Hackathon by Coinbase"

URL: http://blog.coinbase.com/post/76553987867/introducing-bithack-hackathon-by-coinbase


We’re excited to announce the launch of Bithackathon.com – an online hackathon to inspire developers across platforms and continents to build solutions with bitcoin.

 

We will judge app entries based on creativity, usability, and execution. The prize? Bitcoins!

  • 1st prize: $10,000  worth of bitcoin
  • 2nd prize: $5,000  worth of bitcoin
  • 3rd prize: $3,000  worth of bitcoin

Criteria

We are looking for apps that excel in three areas:

1) Creativity:

  • Originality of idea
  • Innovation

2) Usability:

3) Execution:

We invite any and all developers around the world to participate in the competition.  Check out www.bithackathon.com for more information – spread the word!  Look forward to seeing you there.

Original post on Hacker News

usablica/progress.js · GitHub

Comments: "ProgressJS – A themable HTML5 progress-bar library"

URL: https://github.com/usablica/progress.js/


ProgressJS

ProgressJs is a JavaScript and CSS3 library which help developers to create and manage progress bar for every objects on the page.

How To Use

1) Include progress.js and progressjs.css in the page (use minified version from minified folder for production)

2) Execute following JavaScript code in the page:

//to set progress-bar for whole page
progressJs().start();
//or for specific element
progressJs("#targetElement").start();

Use other methods to increase, decrease or set a auto-increase function for your progress-bar. Furthermore, you can change the template using setOption method.

API

Check the API and method usage with example here: https://github.com/usablica/progress.js/wiki/API

Build

First you should install nodejs and npm, then first run this command: npm install to install all dependencies.

Now you can run this command to minify all static resources:

make build

Roadmap

  • Add example folder and provide examples
  • More browser compatibility + mobile/tablet device support
  • Add more templates

Release History

  • v0.1.0 - 2014-02-14
    • First version
    • Increase, decrease and auto-increase functions
    • Ability to design and add templates

Author

Afshin Mehrabani

License

Copyright (C) 2012 Afshin Mehrabani (afshin.meh@gmail.com)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Original post on Hacker News

Fullstack Academy - Romance.js - Programming Valentine's Day Poetry

Comments: "Fullstack Academy - Romance.js - Programming Valentine's Day Poetry"

URL: http://blog.fullstackacademy.com/post/76605703216/romance-js-programming-valentines-day-poetry


On this Valentine’s Day - two aspiring poets at Fullstack Academy want to help you use programming to further your own romantic endeavors.

A bit of background

Nimit and I (David), now instructors at Fullstack Academy, actually met on the first day of college over 14 years ago and bonded over our mutual love of Web technologies and the ethos of open-source (our youthful form of rebellion consisted of running Linux instead of Windows). That’s our founder story and why we continue to this day to love working with the Web and training aspiring Web developers. However, what’s less well known is that we also share a mutual love of poetry and considered ourselves somewhat accomplished amateur poets throughout college.

Although our poetry career never fully blossomed, on occasion you might still find one of us counting syllables on fingers or looking up rhymes as we construct a poem for our loved ones. I’ve always found the hardest thing to do is find inspiration - find something that knows what I want to say and how I want to say it.  Fortunately, with a bit of computer programming, we can train our computer to mimic any poet, including ourselves.

If you’ve started with Codecademy or Codeschool’s JavaScript course, this exercise should be right up your alley. It’s a fun way to explore language and we hope it will inspire you this Valentine’s Day.

Sounding like ourselves

If you’ve ever played Mad Libs, you’ve experience the amusing quality of text interpolated with some randomness. Our mind has an amazing ability to impose a story onto that randomness. Our poetry program is like Mad Libs but with two differences:

Our program chooses every word rather than filling in blanks Our program trains itself by building a set of all words that follow each word in the corpus (the corpus is the poetry or text you want to mimic). Our program picks a random word from the corpus to start. Then it picks from the set of words that have followed this word.

For example, if we take one of my favorite poems from e.e. cummings:

since feeling is first
who pays any attention
to the syntax of things
will never wholly kiss you;

wholly to be a fool
while Spring is in the world

my blood approves,
and kisses are better fate
than wisdom
lady i swear by all flowers. Don’t cry
—the best gesture of my brain is less than
your eyelids’ flutter which says

we are for each other: then
laugh, leaning back in my arms
for life’s not a paragraph

And death i think is no parenthesis

We first identify the unique words that appear in the poem: “since”, “feeling”, “is”, “first”, “who”, “pays”, “any”, “attention”, etc. Then for each word, we identify the list of words that follow that word. For example, for the word “is” we see “first”, “in”, “less”, “no” and for the word “my” we see “blood”, “brain”, “arms”. We store these “following” words associated with each starting word.

The magic about this is that based on how we train our model (what body of text we put in and what sequences it has) it can create a pretty good approximation of what the author of that corpus sounds like.

Although you can be the judge of whether or not you like the poetry this generates, it does pose an interesting philosophical question: are we anything more than the aggregate of all our experiences and language combined in some probabilistic web.

Setting up the Corpus (poetry you want to mimic)

For our code to work, it first needs some body of text to train itself on.  Let’s set up a place in our HTML where we can dump a bunch of text.  For that, we’re going to use a trick that web developers use to store long strings in HTML - script tags that aren’t JavaScript.  By setting it up this way, the browser ignores it but makes it still easy for us to grab the text out. Feel free to follow along by opening your own JSFiddle.net fiddle.

Now let’s set up a way to store all the pairs.  For our implementation, we’ll create a JavaScript object where the keys are single words and the values are an array of all the words that follow that word in our corpus.  Every time we see a new word combination, we’ll store the combination in our object for use later.  We’re going to store duplicates as well so later on, we don’t have to worry about storing probability weights, we just need to randomly choose a word and the probability will work itself out.

Great, now we have two helpful pieces of data: words and wordpairs.  words is a long list of every word in our corpus and wordpairs is where the pairs are stored.  wordpairs[“love”] will return an array (a list) of every word that our program has ever seen follow love.

Making Poetry

Now that we’ve prepped the kitchen we just need to flambe the paper (don’t worry, your program will botch its metaphors even worse). Let’s write a function called “writePhrase” that takes a length of words and returns a phrase of poetry.  We also write a helper function to randomly choose from an array since we’ll be doing that several times.

Take a look at this following JSFiddle (set up with Shakespeare’s Sonnets) to see what kind of output it creates. Open the JSFiddle  and you can change out the corpus and see what kind of other poems you can create! Perhaps you can use it as an inspiration for a beautiful poem for your own loved one.

Further Challenges

The core system that we’ve built is called a Markov Chain (read more on Wikipedia). It’s a simple and elegant way to model various problems in Computer Science and as we’ve seen, can be used to generate amusing text based off a training corpus. However, there are still many directions to go from here:

  • Our program doesn’t handle sentence structure in the original corpus, this makes it start and stop each phrase awkwardly
  • We’re pretty aggressive about removing punctuation, this produces a bland ending text - try to keep two versions of the word, one that has been normalized and one that keeps the original formatting


If you’d like to continue exploring programming and dive deeper into computer science and web development, feel free to reach out to me at david@fullstackacademy.com. We’re hosting a free one-day introduction to web programming called Road to Code in early March at Fullstack Academy where beginners learn to do other awesome things with code!

Original post on Hacker News

Crazy ants take on fire ants and win | Ars Technica

Comments: "Crazy ants take on fire ants and win | Ars Technica"

URL: http://arstechnica.com/science/2014/02/crazy-ants-take-on-fire-ants-and-win/


Fire ant, meet crazy ant.

If you’ve ever been attacked by red imported fire ants, you can likely attest that these tiny insects have a nasty defense mechanism. When threatened, fire ants inject or dab their enemy with a potent neurotoxic venom that quickly dispatches most other ants and can sometimes even send humans to the hospital.

This deadly assault is a pretty effective deterrent to most other ant species, which tend to keep their distance. But the fire ants have now run up against another invasive species that's willing to take them on: the tawny crazy ant. This unassuming little ant has the surprising habit of aggressively barreling right into a colony of fire ants, seemingly undeterred by their toxic venom.

In this week’s issue of Science, researchers from the University of Texas at Austin report that crazy ants are actually able to detoxify fire ants’ venom, helping them displace this usually dominant species.

A crazy ant’s headlong charge into a mass of fire ants almost always results in it being smeared with venom. But once covered in the neurotoxin, these ants perform a systematic and ritualized behavior. First, an ant stands on its hind legs and curls up its abdomen, touching its mandibles to a small gland at the tip of its body called the acidopore. The ant then runs its front legs through its mandibles and grooms itself fastidiously.

It looks like the ant is covering itself with a secretion from its own body as a response to the fire ant's venom. To determine whether this behavior actually counteracts the venom’s effect, the researchers ran a controlled test. They used nail polish to seal the acidopore of one group of crazy ants, and simply sham-treated a second control group. After coming into contact with fire ant venom, the crazy ants with the sealed acidopores—which could not secrete any chemical defenses—had a survival rate of just 48 percent, whereas 98 percent of the control group survived. Clearly, something originating from the acidopore was increasing the survival of ants covered in venom.

By testing crazy ant secretions, the researchers found that the life-saving substance was actually formic acid from the crazy ants’ own venom. So far, it's unclear exactly how this chemical detoxifies fire ant venom. It’s possible that formic acid denatures the enzymes that enable neurotoxins to enter cells.

While this rare ability confers a huge advantage for crazy ant survival, its biggest implications are ecological. Ever since fire ants were imported into the southern US in the 1930s, they have been the dominant ant species in most grassland ecosystems. But crazy ants—introduced only about 12 years ago—are now taking over, thanks in part to their ability to detoxify fire ant venom. When the two species fight over food or space, crazy ants come out on top 93 percent of the time.

Digging into these two species’ past sheds light on this asymmetry. Tawny crazy ants and red imported fire ants share an evolutionary history since their native ranges overlap in parts of South America. Their arms race began there, with fire ants evolving venom to defend themselves and crazy ants evolving a detoxification mechanism as a counter-defense. Now the chemical warfare has been re-engaged here on a second continent, playing out across the Gulf Coast. And for a second time in the past century, a new invasive ant species is dominating and drastically transforming ecological communities.

Related PSA: crazy ants are attracted to electronics. So if you're a technology lover living in the southeastern US, watch your computers and appliances because these tiny invaders are headed your way.

Science, 2014. DOI: 10.1126/science.1245833  (About DOIs).

Listing image by University of Texas

Original post on Hacker News

Airbnb and Housing - The Airbnb Public Policy Blog

Comments: "Airbnb and Housing"

URL: http://publicpolicy.airbnb.com/airbnb-housing/


This morning, you might have seen a story on Airbnb and housing prices in Marfa, Texas, and beyond. We strongly believe that Airbnb makes communities more affordable and we know it has already helped families stay in the home and community they know and love.

First, let’s start with some context. As readers of this blog know, Airbnb has more than 500,000 listings around the world. If you’re looking for a place to stay in Marfa this evening, there are a total of five options available.  In the six year history of Airbnb, only 17 properties have ever been booked in Marfa. So it’s difficult to argue that Airbnb is having any impact on the housing market in this community.

But many have asked an interesting question about whether home-sharing has an effect on housing prices in communities where more people share their homes. We believe home-sharing makes it possible for more people to pay their bills and stay in their own homes. When Airbnb first got started, we heard from hosts who told us how sharing helped them pay their bills and avoid foreclosure or eviction.

Then, we started doing some research and the data shows that people depend on Airbnb to help pay their bills. We conducted a series of studies in cities around the world. Here are just a few of the key data points:

  • 56% of Airbnb hosts in San Francisco said they use their Airbnb income to help pay their mortgage or rent.
  • 46% of Airbnb hosts in Paris said they use income for essential living expenses such as rent and mortgage payments.
  • 87% of Airbnb hosts in New York share only the home in which they live. And 62% of Airbnb hosts said Airbnb helped them stay in their homes.

The sharing economy is relatively new, so there isn’t much academic research on this question, but noted UC Berkeley Professor and housing expert Ken Rosen conducted a detailed study for us and asked whether home-sharing is responsible for rising rents in San Francisco. Here’s what he wrote:

In cities like San Francisco, the cost of a home or an apartment is never far from anyone’s mind and there is no shortage of theories about why rents continue to rise. One of the latest theories posits that the “sharing economy” and short-term rentals are to blame for high rents. But not only is there no evidence to suggest that short-term rentals are making homes less affordable, our research and analysis indicate that home sharing has the potential to make urban housing more affordable for more families.

Rosen’s work also examined whether some people would stop renting their apartments to permanent residents and start sharing them only on Airbnb. Here’s what he found:

Although there have been media reports of renters and landlords benefiting financially from converting traditional apartments to dedicated short-term rentals, we believe this represents isolated cases of individuals and not the market as a whole.

Data shows that most Airbnb hosts occasionally share only the home in which they live and use the money they earn to help make ends meet. That makes home-sharing an important lifeline that helps families around the world afford to live in the city they love.

Original post on Hacker News

Useful Learning Resources For Web Designers | Smashing Magazine

Comments: "Useful Learning Resources For Web Designers | Smashing Magazine"

URL: http://www.smashingmagazine.com/2014/02/14/learning-resources-roundup/


Fortunately, learning is not limited to only a small minority of people anymore; it is not even limited to visiting a school or a university. The Internet makes it possible for us to distribute knowledge at a small price, and is full of resources to expand everyone’s knowledge on an enormous variety of topics.

Since learning is a lifelong task that doesn’t stop after pursuing a certain academic certificate, this round-up is not only dedicated to beginners. It’s for everyone who wants to become an expert in a certain field or is simply curious about the Web and the latest tools and techniques around them.

We hope that this round-up will bring you closer with many of the valuable resources that are available out there. Some are completely free while others can be obtained through quite affordable memberships. You may also be surprised to find that your local college or university is also publishing free classes and courses on all sorts of topics you can think of — make sure to keep an eye open!

Here are the topics of learning resources we’ll cover today:

Coding

Code Avengers
Code Avengers’ courses are a fun way for students to learn computer programming and Web design. Participating in interactive coding challenges, bug hunts and robot missions takes the pain out of learning JavaScript, HTML5 and CSS3. The lessons are designed for students of all ages.

Coderwall
Thousands of developers share their achievements and current projects on Coderwall. Connect with them, share your own pro tips, and get feedback. Learning new programming languages, tools and technologies has a playful twist and is far from boring. Discover how others are solving their challenges, and gain badges for every completed project.

The Starter League (formerly Code Academy)
Trying to build Web apps without the necessary support and guidance can be painful. The Starter League provides you with expert instruction and mentorship. They also paired up with 37signals (creator of Basecamp and Campfire) to provide hands-on knowledge on building successful Web apps.

Pluralsight
Pluralsight was created by a group of developers with the goal of providing the highest quality training possible — for developers, by developers. The library is huge, with courses on nearly every topic you could ask for.

TekPub
TekPub was created with the goal of educating developers. The mini-casts topics range from JavaScript to Ruby on Rails, with more in-depth tutorials available. (Note: Tekpub has been acquired by Pluralsight; old videos can be accessed via Pluralsight).

PeepCode
PeepCode offers high-quality one-hour screencasts on development resources. Learning the most important techniques is quick and easy with these tutorials. (Note: Peepcode has been acquired by Pluralsight; old videos can be accessed via Pluralsight).

Hacker Rank
On Hacker Rank, coders compete to beat high scores by solving little challenges accross the whole science universe, including areas like algorithmy, code gold or artificial intelligence. How do you match up against others when coding a bot to win at tic-tac-toe?

Mozilla Webmaker
Mozilla Webmaker wants to help you move from merely using the Web to making something amazing with it. It has new tools for you to use, projects to help you get started, and a global community of creators — educators, filmmakers, journalists, developers, youth — all making and learning together.

Google Developers University Consortium
The Google Developers University Consortium features courses on mobile and Web development. You will find many usful resources, especially if you’re working with Android and Google Maps.

Android Training
On Android Training, you will find a collection of classes to help you build Android apps. The classes explain the steps to take in order to solve a problem or implement a feature, using code snippets and sample code.

Programr
Programr is an online lab for students and enthusiasts who want to learn programming. The platform’s goal is to give you the skills you need in order to write sophisticated programs. Test your coding skills by mastering assignments, build your own app right in the browser, and check out contributions by other users. You can even show off your know-how in programming contests. It supports coding languages for console, Web and mobile.

Learn Code the Hard Way
“Less talk, more code” is the credo of Learn Code the Hard Way. Students start by getting code to work; learning the theory comes second. The website focuses on practice by featuring exercises and rote repetition, which will help you gain confidence in whatever you want to do.

Dash
Dash teaches HTML, CSS, and Javascript through fun projects you can do in your browser.

Hack Design
Hack Design is an easy-to-follow design course. You can simply receive a design lesson in your inbox each week, which is hand crafted by a design pro. A great resource if you love learning at your own pace. No fake projects.

Code School
Code School teaches Web technologies in the comfort of your browser with video lessons, coding challenges, and screencasts.

Codecademy
At Codeacademy, you can build your own projects and learn how to create interactive websites, games, and apps. You can also join groups to code with your friends and show off your progress with points and badges.

Codewars
At Codewars you can challenge yourself on kata, created by the community to strengthen different skills. Kata are ranked to approximate difficulty. As you complete higher ranked kata, you progress through the ranks so Codewars can match you with relevant challenges.

LearnStreet
LearnStreet makes learning to code easy for anyone. Whether you would like to start off with JavaScript, Ruby or Python, the platform has got you covered. Build your skill base in the interactive courses, and use it in fun little projects.

PHP Academy
PHP Academy provides free tutorials on PHP and other Web development topics, including MySQL, JavaScript (including jQuery) and CSS. There are free video tutorials, optional premium membership and a forum to ask for help.

PLAYterm
PLAYterm is a platform where CLI users share their skills and inspire others. It can replay your terminal sessions online, and it provides embed codes that you can put on your website. Share your knowledge and help others improve their skills.

The New Boston
With his project, The New Boston, Bucky Roberts makes high-quality education available for everybody. His YouTube channel features a lot of computer-related tutorials on topics such as Java, C++, After Effects or Phyton.

gotoAndLearn
gotoAndLearn is a free video tutorial resource for Flash, Game development and HTML5 by game developer evangelist Lee Brimelow.

repl.it
On repl.it, you can explore more than 15 programming languages in your browser — even on the go with your phone or tablet. Just type an expression into the console and wait for the results. The Web application also lets you save your session and share it with others.

The Pragmatic Bookshelf
The Pragmatic Bookshelf’s mission is to improve the lifes of developers by offering text books, audio books and videos for training. The content is produced by programmers for programmers, addressing relevant cutting-edge topics.

HTML and CSS

30 Days to Learn HTML and CSS
Do you want to learn HTML and CSS but don’t know where to start? 30 Days to Learn HTML and CSS is a free course consisting of one video daily for 30 days. All you have to do is spend 10 minutes a day on a new topic. By the end, you’ll have the necessary building blocks to code an entire website.

A Beginner’s Guide to HTML & CSS
This simple and comprehensive guide will help novices take their first steps in HTML and CSS. Outlining the fundamentals, it teaches you all of the common elements of front-end design and development.

Don’t Fear the Internet
For those who don’t want to learn to code, this website provides a brief introduction to WordPress, CSS and HTML, even throwing in some PHP trickery. Jessica Hische and Russ Maschmeyer have prepared short chunks of technical information in concise videos.

JavaScript

Backbone Screencasts
If you’re learning Backbone.js, these screencast will be very useful. The extensive live coding sessions will walk you through, from the beginning to more advanced stuff like using Backbone.js with Ruby on Rails.

appendTo
JavaScript and jQuery skills are becoming more and more sought after. By offering a number of lessons, appendTo helps you develop those skills. Signing up isn’t even necessary; just watch the free screencasts (each around 10 minutes long), and start building your foundation of JavaScript and jQuery knowledge.

JavaScript Garden
JavaScript Garden is a growing collection of documentation about the most quirky parts of JavaScript. It offers advice on avoiding common mistakes and subtle bugs, and it lays out performance issues and bad practices that JavaScript programmers might run into on their journey to the depths of the language. The resource is dedicated to professional developers, rather than beginners, and it requires some basic knowledge of the language.

NodeSchool
NodeSchool offers interactive lessons for Node.js, including core concepts and electives. There is also a list of free/affordable in-person NodeSchool events around the world which are .

Eloquent JavaScript: A Modern Introduction to Programming
The book “Eloquent JavaScript” by Marijn Haverbeke introduces the JavaScript programming language and programming in general. A free digital version is available in HTML format and you can order a paperback version from Amazon. Furthemore, the book has been translated to French, German and Polish. (Note: A second, more modern edition is currently in process.)

Node Tuts
Node Tuts by Pedro Teixeira offers free webcasts exclusively about Node.Js.

Ruby on Rails

Ruby on Rails Tutorial
Michael Hartl has written a tutorial book named Ruby on Rails Tutorial. Visit the website to find the free online version, along with screencasts.

TryRuby
Ruby is a revolutionary programming language from Japan known for its simplicity and power. On TryRuby, you can experiment with it right in your browser. A 15-minute interactive tutorial will help you get started.

Hackety Hack
Hackety Hack teaches you the basics of programming by introducing the Ruby language. Build your skills from scratch, and use them to build desktop applications and websites.

Virtuous Code
Avdi Grimm is a software “cultivator” who publishes a screencast series on Ruby development. When you subscribe, you get new screencasts every Monday and Thursday (or access to the full scripts if you prefer reading to watching), the source code for each video and access to the complete archive of episodes.

RubyMonk
RubyMonk is an interactive Ruby learning plattform. You will learn the basics of the programming language right in your browser. The tutorials are free but donations are very welcome.

Rails for Zombies
Learn Ruby the zombie way with Rails for Zombie. You don’t have to worry about configuration. After watching short introductory videos, you can start experimenting right in the browser. The course is aimed at beginners, but there are also courses for more experienced Ruby developers.

RailsCasts
RailsCasts by Ryan Bates, offers a free weekly screencast featuring Tips and Tricks for Ruby on Rails. The topics are targeted for intermidiate users, but beginners and experts can also get something out of it. You may subscribe for additional screencasts.

Design

Drawspace
Drawspace is a community of drawing enthusiasts, professional artists and art educators. It features a huge library of free downloadable lessons that teach you how to draw or enhance your current abilities. With a profile, you can track your progress, from beginner to advanced levels.

Miscellaneous Lessons From The Web Industry

Treehouse
Treehouse is for beginners and experts. It offers material for learning everything you need to be successful in the Web industry. This includes technical knowledge, but also skills for starting a successful business. You can learn via specific tracks (i.e. Web developement) or topics.

Tuts+ Premium
Tuts+ Premium is a subscribers-only platform that offers resources for learning creative and technical skills such as design, Web development, motion graphics and photography. The content is created and constantly revised by leading experts. Choose whether to learn by video or articles with screenshots. A large community is behind Tuts+ Premium that you can connect with and ask for further help.

Ontwik
Ontwik gathers the latest lectures and conferences from Web developers and designers in one place. It covers topics such as JavaScript, NodeJS, jQuery, Ruby, Rails, HTML5, CSS3, UI, UX and more. There are also lectures on creativity, marketing and startups.

Because technical knowledge is not enough

A Student’s Guide to Web Design
Here is an attempt to better equip graduates in the design industry. It provides resources and information to help young Web designers with life after graduation.

#The50
After graduating from art college, Jamie Wieck realized that he had no clue about professional life. So, he started #The50 to help students and graduates in the same situation learn what every creative should know. The tips are made up of 140 characters and a hash tag, making them easy to share on Twitter.

The Web Design Community Offers Advice to Beginners,” Smashing Magazine
We asked, “What is the single best tip from your experience that you would give to a newbie developer?” This article compiles all of the amazing responses we received.

Jessica Hische’s Thoughts
Illustrator Jessica Hische doesn’t have a traditional blog, but she shares answers to frequently asked questions about her and her work. You’ll find useful advice on random topics regarding the Web industry such internships, pricing, non-creepy networking, and so on.

The Secret Handshake
The creative industry is very different from traditional companies and applying only traditional methods in the application process won’t bring you too far. The Secret Handshakes is a resource for students and young creatives looking for insiders insights, honest answers and solid solutions to help you go pro.

WaSP InterAct Curriculum
Designed to keep up with the fast-moving industry, WaSP InterAct is a living curriculum that prepares students for careers on the Web. Courses are divided into several learning tracks, covering everything from the foundations to professional practice. Recommended reading lists, assignments and exam questions help you to become a real Web professional.

Conference Organiser’s Handbook
Are you planning to organize a conference? Then, the Conference Organiser’s Handbook is the best place to start. The website was put together by Peter-Paul Koch and provides information on everything you need to know, from start to finish.

Expanding Your General Knowledge

TED
TED is devoted to “ideas worth spreading.” You can watch talks on technology, design, business, global issues, science and entertainment. Get inspired by other thinkers, and get involved in a community of curious people!

Khan Academy
The Khan Academy wants to provide anyone anywhere in the world with a world-class education — for free! All you need is an Internet connection. A huge video library provides you with lessons on a wide array of topics, covering everything from basic maths to macroeconomics to programming basics and even art history. Test your knowledge, track your progress and earn badges.

University of Reddit
The University of Reddit is an open-source peer-to-peer learning platform. The courses are free and range from computer science to mathematics to languages.

VideoLectures.Net
Registering on this site gives you free access to educational video lectures. The videos cover many fields of science and feature distinguished scholars and scientists at conferences, workshops and other events. The high-quality content is aimed at both the scientific community and the general public.

P2PU
The Peer 2 Peer University is an open-education project that encourages lifelong learning, following its credo “We are all teachers and we are all learners.” Everybody can participate and make use of the content. It also features a School of Webcraft, with some basic tutorials.

Online Courses
Online courses offers 100 open courses for tech geeks. Among them, you will find general computer science topics and courses on Web design and development. The website also provides information on accredited schools, college finances and studying.

Lynda
Lynda helps you learn software, creative and business skills. As a member, you get unlimited access to a huge library of high-quality video tutorials, taught by working professionals. Topics also include design and development.

Learners TV
Learners TV provides a huge collection of free downloadable video lectures on all sorts of topics, including computer science. The website also features science animations, lecture notes and live, timed online tests with instant feedback and explanations.

ReadWrite
ReadWrite covers all things Web, tech and social media. Its list of tech-focused instructional websites links you to platforms that teach a wide array of topics. The topics are pretty general, ranging from computing to hacking.

Learn a new language

Radio Lingua
Radio Lingua is a podcast that helps you learn languages where, when and how you want. There are quick starter courses if you want to learn the absolute basics of a language, or you can take your skills to the next level by diving into grammar and vocabulary. The episodes are aimed at learners of all ages and conducted by experienced teachers and native speakers.

Busuu
Learning a language with Busuu is completely different from what you are used to. As a member of the platform’s community, you learn directly from native speakers via video chat. That way, everyone is not only a learner, but also a teacher. To keep you motivated, the language you are learning is represented as a tree, which grows with the effort you put in. Joining Busuu is free.

Open University classes & University-style classes

Udacity
Udacity’s learning experience is different from other education platforms. Learn by solving challenging projects and by interacting with renowned university instructors and other students. The courses are as demanding as studying at a real university, but a range of certificate options are available.

OnlineCourses (formerly know as Lecturefox)
On OnlineCourses you will find high-quality classes from renowned universities such as Harvard, Berkeley and MIT. Topics range from biology to accounting, foreign languages to science.

Education Portal
Making education accessible is the goal of the Education Portal. The platform offers articles and videos on researching schools, degree programs and online courses. Covering everything from arts to sciences, it also has a list of free Web design courses that lead to college credits.

OpenClassroom
Stanford University’s OpenClassroom provides videos of computer sciences courses. You can watch the videos for free, and the lessons are split up into short chunks of quality information.

MIT OpenCourseWare
MIT OpenCourseWare publishes virtually all MIT course content. The open platform doesn’t require any registration, and it features free lecture notes, exams and videos.

OpenCourseWare
The OCW consortium is a collaboration of higher-education institutions and associated organizations from around the world to create a broad and deep body of open-education content using a shared model.

The Faculty Project
The Web isn’t be the only thing you are interested in. If so, then the Faculty Project might be for you. It brings lectures from leading university professors to anyone with an Internet connection. The free courses are taught through video, slideshows and reading resources, and they cover lessons from maths to economics to history.

Academic Earth
Whether you want to advance your career or just take classes that interest you, Academic Earth provide anyone with the opportunity to earn a world-class education. The website offers free lessons and learning tools from many disciplines. If you would like to study further, it also connects you to universities and scholars.

Course Hero
Course Hero has a mission to help college students get the most out of their education by giving them access to the best academic content and materials available. Search for study documents by department, keyword and even school. After registering, you can use the resources for free.

edX
edX is a not-for-profit enterprise by MIT, Harvard, Berkley and the University of Texas System. Take part in high-quality online courses from different disciplines — including computer science — and obtain a certificate from one of the renowned universities. The institutions use the insights they gain from the platform to research how technology can transform learning.

Coursera
Partnering with the top universities from around the world, Coursera offers free online courses. The lectures are taught by renowned professors and cover a variety of disciplines. Assignments and interactive exercises help you test and reinforce your knowledge.

Webcast.berkeley
Since 2001, Webcast.berkeley has been a window into UC Berkeley’s classrooms, publishing courses and campus events for students and learners everywhere in the world. View audio and video recordings of lectures online, or download them to your device.

The Open University
The Open University is open to anyone and offers over 570 courses at many different levels, from short introductory courses to postgraduate and research degrees. Studying is flexible and adapts to your lifestyle. You can even connect to other learners online and use the activities to assess your progress.

Last Click…

WeekendHacker
Do you have a small project or idea in mind but need a little help? WeekendHacker is a place to reach out to designers and developers who may be able to assist. Simply sign up, post your project, and sit back and wait for someone to help.

Until Next Time!

We hope that this list of learning resources will help you to further develop your skills and open doors for you. Of course, you’re more than welcome to share other resources that are missing in this round-up in the comments section below! Also, we look forward to hearing which resource you find most valuable, and why!

By the way, you may also want to check out Melanie Lang’s list of inspirational podcasts — we highly recommend it!

(sh, ml, ea, il)

Front page image credits: Programmr.

Advertising

Original post on Hacker News

How I want to write node: stream all the things!

Comments: "How I want to write node: stream all the things!"

URL: http://caolanmcmahon.com/posts/how_i_want_to_write_node_stream_all_the_things_new/


I wrote the async library back when Node first removed Promises from core (yes, that really was the case). Back then, I preferred to do with plain callbacks what was usually done by including third-party Promise, Future or Continuable libraries instead.

I find a certain elegance in describing complex patterns using simple parts, particularly when the parts are made composable through a common interface. In Node, and JavaScript in general, you're likely to use a combination of Promises, callbacks, Streams, Event Emitters and even ES6 Generators. To me, however, these all represent values in the future. What if we could reduce entire programs to transformations over a Stream, and have just one API to rule them all?

Well, here is my proposal, for your consideration and feedback:
Highland, a high-level streams library

var _ = require('highland');
var doubled = _([1, 2, 3, 4]).map(function (x) {
 return x * 2;
});
var data = filenames.map(readFile).parallel(4);
data.errors(function (err, rethrow) {
 
});
data.pipe(output);
var output = fs.createWriteStream('output');
var docs = db.createReadStream();
_(docs).filter(isBlogpost).pipe(output);
docs.pipe(_().filter(isBlogpost)).pipe(output);
var clicks = _('click', btn).map(1);
var counter = clicks.scan(0, _.add);
counter.each(function (n) {
 $('#count').text(n);
});

This is not a new idea, but I believe it is a new combination of features, which is important. Those of you that work with Streams in Node may be aware of the excellent modules by Dominic Tarr, similarly if you're from the browser you might use FRP libraries such as RxJS. These are all great, but they hint at a deeper abstraction, one which would allow us to write entire programs using Streams. In an attempt to achieve this, Highland implements:

  • Back-pressure support - Data sources are regulated so that slow consumers are not overwhelmed
  • Laziness - So we can use the reading of Streams to sequence the execution of code - this means we can choose to read from files in parallel or series, or stop reading after an error, for example
  • Asynchronous operations - So we're able to handle async data sources and async transformations
  • Error and data channels - So we can manage error propagation from sync and async code
  • Compatible with Node Streams - So we can pipe Node Streams to and from Highland Streams and play nicely with the Node ecosystem

If you find this idea intriguing and want to explore it further, then check out the Highland website. I'd love to hear your experiences.

Highland on GitHub

Original post on Hacker News

solo.im - a single founder peer group

Comments: "solo.im - a single founder peer group"

Original post on Hacker News

doo | Every ending holds a new beginning

Comments: " doo | Every ending holds a new beginning "

URL: https://doo.net/story.html


Every ending holds a new beginning

It is with a heavy heart we have to announce that “doo – the document app” will be shutting down on Monday, March 17, 2014.

We founded this company in June 2011 to change the way people organize and work with documents. These were our hypotheses.

  • We believed the time had come to shift from “folder-based” to “tag-based” organization of documents.
  • We believed the increasing number of digital documents and document sources poses a big problem to many people.
  • And we believed people are struggling to access all these documents and document sources from their (mobile) devices and to keep them “in sync”.

Well, maybe we’re still too attached to our “baby”, but we actually still believe in these hypotheses. Anyway, here are the facts.

  • We launched the first doo app for Windows 8 in October, 2012 – right on time for the Windows 8 launch. The app gave us a tough time – also in subsequent months – since it was really hard to do what we do on this platform.
  • In parallel we got our data intelligence team up and running and started to build advanced data intelligence technology for documents including image pre-processing, OCR and auto-tagging based on a combination of machine learning- and several other algorithms.
  • Then, in February, 2013 we launched doo for OS X. Apple featured the app as “editor’s choice” on the Mac App Store. All major tech blogs covered us. doo for OS X soon became top downloaded app #1 globally and in more than 50 markets including US, UK, and Germany. But after a lot of initial downloads, we soon saw a significant drop in our download and user activity figures. “Well, the app is not feature complete, performance needs to be optimized either and we don’t have the mobile apps yet to make our case regarding sync. It’s OK”, we thought and moved on.
  • In March, 2013 we received the Innovate 4 Society award sponsored by Microsoft. The award was presented by Chancellor Dr. Angela Merkel to our CEO, Frank, at the CeBIT.
  • Next, we launched our Android app in April and iOS app in August, 2013. Now, we had our mobile apps including mobile scanning. The Android user base displayed steady growth and user activity; plus, we received a lot of positive user ratings on Play and were selected as a TopApp by Deutsche Telekom. However, we couldn’t get enough visibility on Play to accelerate our growth.
  • The iOS app, in contrast, received a lot of attention. It even became one of the “Best Apps 2013” on the iTunes Store later this year. And again, same experience as doo for OS X.
  • We were very happy to see a world class company like Evernote feature our app.
  • In October 2013 we finally launched a beta version of our Windows Desktop app; and the RC in January 2014.

After launching all these apps, delivering our core feature set and continuously improving on performance and stability, we still couldn’t generate the necessary growth and user activity. We had to face it. Either our hypotheses were wrong or we didn’t get the product right to address the problem. We simply do not seem to address the needs of a large enough audience.

Thus, as of Monday, March 17, 2014 at 4pm CET the doo servers will be shut down and the User License agreement for the use of doo is terminated with effect as of this date. The doo apps for OS X and Windows will continue to work locally, but the apps for Android, iOS as well as all features related to our server such as synchronisation among devices, connecting and synchronizing with cloud services, or online login to the doo user accounts will no longer work.

All user data and files on doo servers will be completely and irrevocably deleted. All data in your local apps will, of course, not be affected. Have a look at our user guide and FAQ for more detailed information regarding the shut down.

So what’s next?

Even though we’ve been chasing the wrong rabbit, we have built outstanding cloud storage, sync and search technologies. Plus, we have great investors, who believe in our team. So there is only one option: we have already started working on new products based on what we’ve learned … we’ll keep you posted!

Thanks a lot for all your feedback, encouragement and support! And don’t hesitate to reach out with any thoughts or questions. Email us at hello@doo.net or tweet @doo.

Original post on Hacker News

- 页面 2 的 3 -