Changelog

Follow up on the latest improvements and updates.

RSS

improved

AuraDB Professional

AuraDB Free

Data Importer

Import

Data Importer - Preview and Files panel updates

This release of Data Importer brings a number of small changes that make using Data Importer that little bit easier :
Preview graph visualisation
The Import preview is now the last part of Workspace to migrate from the old visualisation to the new one used in Explore and Query. As well as being more visually consistent, it's now more responsive and easier to navigate.
CleanShot 2024-01-17 at 11
We'll be exploring upping the limits on the number of preview nodes, do leave us feedback if you think that'd be useful.
Files panel
  • Files are now sorted alphabetically, making files easier to locate in longer lists.
  • Files can now be globally expanded and collapsed making it easier to work with longer lists.
CleanShot2024-01-17at11
That's all for now. We're working on some big new capabilities in import, more on that to follow in the coming months 😀. As always keep your feedback coming on feedback.neo4j.com.

new

AuraDB Professional

AuraDB Free

AuraDS Self-Start

Data Importer

Import

New in Data Importer

This release of Data Importer brings two new features:
Preview selected
We've made a slight tweak to the Preview feature, allowing you to optionally preview selected parts of your our graph. If you have a particularly complex model, it can sometimes be hard to see how all the different nodes and relationship types being previewed are linking. In these scenarios you can use "Preview selected" to only preview selected parts of the model. This can help you validate specific parts of the model are mapped as expected. In the example below, only the Product, Supplier and SUPPLIES parts of the model will be previewed when using the "Preview selected" option.
CleanShot 2023-09-18 at 15
Generate Cypher script
The Data Importer UI gives you a convenient way to generate your graph model and map data, but what happens if you want to take the import logic and use it somewhere else without the UI? Or what if the Data Importer UI doesn't quite allow the complex transformations that you need?
For these scenarios, we are now providing you the option to view and download a Cypher script representing your model and mapping. You can use these as-is, or adapt for more complex needs.
CleanShot 2023-09-19 at 09
CleanShot 2023-09-18 at 15
CleanShot 2023-09-18 at 15
Examples of where you might use this generated code include:
  • The Workspace Query UI. Note that the files will need to be accessible to the DBMS you are connected to. At present this means Aura customers will need their files accessible on a public URL for the
    LOAD CSV
    command to access them.
  • On the command line via cypher-shell
  • Driver code - De-compose the script to re-use the
    CREATE CONSTRAINT
    and
    MERGE
    statements to implement your loads
The load script itself is commented to help you understand the purpose of the different parts and where you might need to make changes, depending on where you're running it.
You can also choose to download the files with your script, which will give you a handy zip of the script and your files.
We hope you enjoy using these new features and as always please head over to https://feedback.neo4j.com/data-importer with any feedback.

new

AuraDB Professional

AuraDB Free

Data Importer

Data Importer - UI refactor and model only support

In this release of Data Importer, we're pleased to announce changes to make the UI clearer, as well as allowing you to use Data Importer as a simple standalone graph modelling tool. Here's a quick run-down of those changes:
Files panel
The Files panel now hides automatically when you don't have enough space to easily view your model and mapping. You can always bring it back temporarily to overlay on you graph model when screen space is limited. You can also tuck it away permanently when it's not needed.
ezgif
Mapping panel
The Mapping panel has undergone a number of changes:
CleanShot 2023-06-21 at 16
  1. The files dropdown will now only be shown when you have added files to the UI. This removes clutter from the UI if you just want to sketch out a graph model.
  2. You can now multi-select properties, allowing you to delete more than one at a time,
  3. The mapping column is now placed immediately next to the property (and type) you are mapping it to. It was previously in a separate mapping tab. The mapping column is also only shown when you have files added.
  4. The ID for a node is now selected by clicking the key icon next to the property you want to be the ID.
  5. There is a new way of entering an editable state for property names. You'll also notice that properties now have a consistent dark grey background, including when they're being referred to in the relationship mapping UI:
CleanShot 2023-06-21 at 16
There are quite a few other small changes made in the UI, we hope you agree they make the UI cleaner and established users don't find them too disorientating. As always, please do head over to leave us your feedback on https://feedback.neo4j.com/data-importer. Thanks!

new

Data Importer

New in Data Importer

Manually stop import
You'll now see that the run button changes to a stop button that also reports progress. You can use this button to terminate the load in the event you need to make tweaks before relaunching the load.
CleanShot 2023-04-27 at 13
You'll also notice that the preview button has changed to a preview icon.
Empty property handling
Previously Data Importer created properties for empty strings, occupying unnecessary storage. From this release properties will no longer be set for empty strings, resulting in smaller database sizes.
Mapping completeness
You'll also notice that incomplete mappings are no longer rendered as dashed lines. Instead you'll now see a green check mark against nodes and relationships that are fully mapped.
CleanShot 2023-04-27 at 14

new

AuraDB Professional

AuraDB Free

Data Importer

Import

Data Importer - Introducing file filtering

This release of Data Importer introduces a new way to load more data sources without the need for pre-processing. By allowing you to apply simple filters to files we're enabling loads in more scenarios, including:
  • Generally keeping data relevant from only certain rows in a file while skipping the others
  • Loading data from aggregate
    node lists
    and
    relationship lists
    where information on all nodes and all relationships is encapsulated in just two files.
We're going to take a quick look at how file filtering can help you with the latter example.
Consider the following subset of the Northwind data model showing Orders, the Products they contain and the Shippers they are shipped by.
CleanShot 2023-01-26 at 11
In the classic Northwind dataset, these are represented by different tables like
Shippers.csv
,
Orders.csv
,
Products.csv
and
Order-Details.csv
. When extracting data from more graph-like sources however, it is not uncommon to be provided with wide node lists and relationships lists that contain all the nodes and all the relationships in just two files.
Here's what an example exported from our very own Neo4j Bloom looks like (but could equally apply to any other graph-like export):
CleanShot 2023-01-26 at 11
bloom-nodes-export.csv
CleanShot 2023-01-26 at 11
bloom-relationships-export.csv
In the nodes file you'll notice the node types are identified by the
~labels
column and relationship types in the relationships file by the
~relationship_type
column.
Prior to the file filtering feature, you needed to separate the nodes files into three files representing the three node types and the relationships file into two files representing the two relationship types.
With file filtering, now optionally available under the File dropdown in the Mapping Panel, you can apply include filters to keep rows only relevant to the Nodes or Relationships in your model. Here's an example of applying the file filter to the
bloom-relationships-export.csv
file to ensure it only keeps the rows where the
~relationship_type
column has
ORDERS
values.
CleanShot 2023-01-26 at 12
You'll notice the filter when applied, gives you feedback as you type on how many matches were found. For performance reasons only the first 10,000 rows of any file are scanned, so even if you don't see matched in this feedback, there may still be matches further down your file.
The same filtering principle applies for the nodes file, in this example mapping to the Product node and only keeping the rows in the
bloom-nodes-export.csv
file where values in the column
~labels
equal
Product
.
CleanShot 2023-01-26 at 12
For now file filtering supports exact string matches, but we'd love to hear your feedback on the utility of the filtering functionality and other things you'd like to see. As always, please head over to https://feedback.neo4j.com/data-importer to leave us your feedback.
That's all for now, thank you for reading!

new

AuraDB Professional

AuraDB Free

Data Importer

Import

New in Data Importer

This release of Data Importer brings a couple of small UI updates to make common tasks easier and clearer.
Collapsible files panel
The files panel is now collapsible, giving you more space to work on the graph model when needed.
CleanShot 2022-12-11 at 20
Reworked Relationship From/To Mapping details
We've observed that it isn't always clear to new users what the different parts of the From/To relationship mapping UI represent, so we've reworked it to hopefully make it a little clearer.
CleanShot 2022-12-11 at 20
We now explicitly call out which nodes and ID properties are at the
From
and
To
ends of a relationship, and which file columns we are expecting you to match them up with to create valid relationships.
We continue to automatically populate these for you when your relationships use the same file for nodes at each end (and you've already selected their IDs). The main time you need to populate these fields manually is when you're using a separate file to define relationships between nodes.
As always, keep your feedback coming on https://feedback.neo4j.com/data-importer.

new

AuraDB Professional

AuraDB Free

Data Importer

Import

Data Importer - latest updates

You'll see a number of small updates land today that make Data Importer easier to use:
  • Cleaner rendering of the Graph Model and drag-out relationship creation
  • Constraints created by Data Importer to perform loads are now prefixed
    imp_uniq_
    so you can easily identify them
  • When you only have one file in your files panel, we'll assume that's the one you want to map and automatically select it in the node mapping panel to save you time
  • When running in Workspace, Data Importer now gives you the option to
    Start Exploring
    in the Explore tab rather than Query after an import completes. Standalone Data Importer continues to provide a link to Query in Browser
  • Cypher statements shown in the Load Summary now have syntax highlighting
Keep your feedback coming on feedback.neo4j.com/data-importer.

new

AuraDB Professional

AuraDB Free

Data Importer

Data Importer - now with DateTime datatype support

This release of Data Importer brings support for the DateTime datatype. You can now update your existing model configuration to use the DateTime datatype or if you're creating new properties from file columns, we will attempt to autodetect them for you.
CleanShot 2022-10-12 at 13
The DateTime support is designed to require minimal user input, so there's no need to specify a date format. Behind the scenes we achieve this by using JavaScript's lenient date parsing which takes a best guess at how to parse the string. You should be aware of the limitations of lenient date parsing, where ambiguous (eg US vs GB day/month ordering) or specific date formats having the potential to lead to unexpected results. If you want to be 100% confident your DateTimes will be handled correctly in Data Importer, you can guarantee this by ensuring your date is in ISO 8601 format
YYYY-MM-DDTHH:mm:ss.sssZ
or one of its variants:
  • 2011-10-10
    (date-only)
  • 2011-10-10T14:48:00
    (date-time)
  • 2011-10-10T14:48:00.000+09:00
    (date-time with milliseconds & timezone)
As always, we look forward to hearing feedback, so please head on over to our feedback board and leave us a post with any comments.

new

AuraDB Professional

AuraDB Free

Data Importer

Data Importer usability improvements

This minor release of Data Importer brings a number of small but useful improvements:
  • Datatype detection
    : For properties created using
    Select from file
    , we now scan the first few rows of a file to make an educated guess on the datatype
CleanShot 2022-09-05 at 14
  • Node ID selection
    : We've tweaked the node id selection logic so that we'll now only choose a column ending
    id
    as a node id if it's the only such column in a file.
  • Problem files:
    Duplicated or empty column names in your files have the potential to cause unexpected behaviour or load errors. We now make it clear which files are affected allowing you to fix them before continuing.
CleanShot 2022-09-05 at 14

new

AuraDB Professional

AuraDB Free

Data Importer

Improved Load Preview and new mapping support

This latest release of Data Importer brings yet more improvements to make it easier to build your graph, read on to find out more...
Higher fidelity Load Preview
Last release we brought you the initial release of Load Preview, which allowed you to get a feel for how the structure of your graph was taking shape before performing a load. We've now made the preview logic more faithful to the load process so any problems with your mapping will now be more accurately reflected in the preview. We now also support the previewing of properties.
High fidelity Load Preview
We hope these big improvements will help you understand how your graph is taking shape quicker than ever.
Automatic mapping of common relationship IDs
When mapping properties from a file, we will now try and guess the id value by looking for a column named
id
or ending in
id
. In the event you map only one property from a file, we'll assume this is the property you want to use as the ID. You can of course override any of the automatic mappings yourself if they are not what you intended.
Automatic ID mapping
Automatic relationship from/to mapping for denormalised files
When mapping the same file to a relationship and the nodes at each end of its ends, it's usually fairly easy for us to deduce which columns should be mapped to the
from
and
to
once you've mapped the IDs to the nodes. We now do this for you automatically and again you are free to override this mapping if needed.
Automatic relationship from and to mapping
Note that this currently only works when you connect two nodes with a relationship
after
the node IDs have been set, but we plan to improve that in future.
We hope these automatic mappings speed up iterating your graph modelling and mapping.
Small bug fixes and improvements:
And finally we also made a number of small improvements as a direct result of your feedback, so please do keep the feedback coming on our feedback page:
  • Resolved an issue where integer values wouldn't load when the flat file column was padded with whitespace.
  • Resolved an issue that would cause loads to fail when a Boolean value was used as an ID
  • Scroll zoom in Load Preview now follows the mouse cursor
  • Files with larger row lengths are now supported
  • Various small UI improvements
  • Updated Cypher statement generation ready for Neo4j 5.0 release
Load More