The Knowledge Graph Conference Icon
The Knowledge Graph Conference
  • 馃彔Home
  • 馃搮Events
  • 馃懁Members
  • 馃數Announcements
  • 馃數Ask
  • 馃數Ask The Ontologists
  • 馃數Events
  • 馃數Jobs
  • 馃數Promotions
  • 馃數Share
Powered by Tightknit
Ask
Ask

Exploring Spark GraphFrames: Limitations and Potential Insights

Avatar of Beegee A.Beegee A.
路Dec 07, 2021 09:17 AM

Posting a question here that is more ephemeral and interactive: Does anyone use Spark Graphfames yet? With the 0.8.0 version it鈥檚 already showing nice potential. The constructor looks limited though with the parameters GraphFrame(v,e) Which appears to limit one type for vertices. e.g. I can easily represent a network of people but not a network of people and articles they read. I suppose I can consolidate various types of vertices in the same dataframe

馃憖3

5 comments

路 Sorted by Oldest
  • Avatar of Rashif Rahman
    Rashif Rahman
    路

    I have very brief experience with Spark and GraphFrames while evaluating a use case for graph analytics at scale, for both LPG (Neo4j) and RDF. You are supposed to "consolidate". Like many other tools that focus on analytics rather than semantics (and also viz such as networkx), these graph-aware data structures are not node-type-aware; the interpretation is left up to you and what you encode in the node attributes. Much of the effort in the graph analytics space has been on "homogeneous" networks, and few on "heterogeneous" ones like a true KG, so there tends to be no special facility for "typed" nodes/vertices..

  • Avatar of Beegee A.
    Beegee A.
    路

    Good to know Rashif Rahman. Thank you

    馃憤1
  • Avatar of Beegee A.
    Beegee A.
    路

    Hello, Rashif Rahman. Do you have a combination of tools that you prefer for analytics, semantics, and visualizations?

  • Avatar of Rashif Rahman
    Rashif Rahman
    路

    Hey Beegee A., I apologize that I don't really have good recommendations for tools that cover all three (analytics, semantics, visualization) well. I believe analytics at Spark scale is tough to match, and you can't really control viz in most DBs. Maybe someone else can chime in.

    :thanks:1
  • Avatar of Martynas J.
    Martynas J.
    路

    Beegee A. https://www.youtube.com/watch?v=UMIVZZiNwm8

    Video thumbnail
    馃憖1