Reporting Overview

Lucy Reporting  


AVAILABLE REPORTS

Currently the reports available for Admins to export are:

  • Question Report - Contains the specific questions asked by the team.
  • Activity Report - Contains activity related to asking questions and actions on results received.
  • Usage Report - Describes usage within an instance.
  • Lucy Assist Report - List of all Lucy Assist actions within an instance.
  • Users Report - An export of all users within your instance along with their associated names, email, and active status.
  • CMS Report - Contains all the activity across the Capabilities, Community and Quick Links pages.
  • Filter Report - Contains all questions asked by users when a filter was applied, and what filter(s) was applied.
  • Training Report - Contains a summary of all training events and questions asked over the specified date range. This is useful for investigating how much training users are applying and to which questions.
  • Indexing Report - Report of all file status for all sources configured within your instance.

Use the days selection box for each report to select the desired number of days and click export. This will export a report of all the activity in your instance within the specified time period. 

Graphical user interface, application Description automatically generated

Figure 1 – View of the Reports section in Settings.

All report exports will appear in the Exports section in Settings and provide Admins with the ability to download the reports. An email will be sent to all Admins once the report export has been generated and is available for download. The Exports tab will also display the status of all exports and allow Admins to delete exports they no longer need. 

A screenshot of a computer Description automatically generated

Figure 2 – View of the Exports section in Settings.


QUESTION REPORT

A Question Report for only the instance you are in or viewing as.

Question Report Elements: 

  • company - The company or instance the user is asking the question in
  • first_name - The user’s first name (option to use non-PII Key)
  • last_name - The user’s last name (option to use non-PII Key)
  • email - The user’s email address (option to use non-PII Key)
  • question - The question the user asked
  • short answer – The synopsis response to the user’s question
  • date - The date asked
  • attribute – If applicable, assigned attribute

A close-up of a question Description automatically generated with low confidence 

Figure 3 - Example Export.


ACTIVITY REPORT

All activity related to asking questions and actions on results received.

Activity Report Elements:

  • company_name - The company or instance the user is asking the question in
  • first_name - The user’s first name (option to use non-PII Key)
  • last_name - The user’s last name (option to use non-PII Key)
  • email - The user’s email address (option to use non-PII Key)
  • group – If applicable the user’s associated group
  • question_id - The unique ID number assigned to every question
  • question - The question the user asked
  • short answer - The synopsis response to the user’s question
  • platform - The platform from which the question was asked
  • filename – The filename from which the answer came from
  • title - The title of the answer received (auto generated)
  • source - The name of the content source presented
  • confidence - What percent confident Lucy is that this is the correct answer
  • events- If the user has interacted with this answer
    • Event Actions:
      • Answer Clicked - The user clicked into this answer
      • Answer Verified - The user has verified this answer
      • Negative Trained - The user clicked thumbs down on the answer
      • Positive Trained - The user clicked thumbs up on the answer
      • Save Clicked - The user clicked save on the answer
      • Saved Answer Removed - The user clicked remove save on the answer
      • Shared Clicked - The user shared this answer and/or copied the link
      • Open Shared Answer - The user opened an answer that was shared with them
      • Source Link Clicked - The user clicked on the source link of this answer
      • View All Pages Clicked - The user clicked view all pages on this answer
      • Copy Text Clicked - The user has clicked Copy Text in this answer
      • Related Question Clicked - The user clicked a related question at the bottom of this answer
      • Synopsis Copy Text - The user clicked Copy Text in the synopsis response
      • Synopsis Expanded - The user expanded the synopsis response
      • Synopsis Positive Feedback - the user clicked thumbs up on the answer
      • Synopsis Negative Feedback - the user clicked thumbs down on the answer
      • Hidden Search Results Expanded - The user opened Related Content (select clients)
      • [Deprecated] Comment Added - The user added a comment to the answer
  • position - The position (1 - 10) the answer was presented in
  • date - The date and time at which the question was asked
  • Attributes – The attributes the user is asking the question is assigned to


USAGE REPORT

This report has 3 tabs which are:

  1. Ingestion Report
  2. User Login Report
  3. User Question Report 

User Ingestion Report Elements:

  • company - The company for which the file was ingested on behalf of
  • sector - The sector/group assigned to the user
  • project_name - The name of the project that the file was ingested into
  • size - The size of the file ingested in bytes
  • pretty_size - The size of the file ingested in MB
  • files - The number of files being ingested
  • folder URL – The URL of the connected folder

User Login Report Elements:

  • login_at - The time at which the user logged in
  • user_id - The unique ID assigned to the user in Lucy
  • user_email - The email of the user (option to use non-PII Key)
  • sector – If applicable, the sector/group assigned to the user

User Question Report Elements:

  • asked_date - The date on which the question was asked
  • question_asked - The question that was asked by the user
  • user_email - The email of the user (option to use non-PII Key)
  • user_id - The unique ID assigned to the user in Lucy
  • sector - The sector/group assigned to the user


LUCY ASSIST REPORT

This is a list of all Lucy Assist actions within an instance. If a user reported finding what they are looking for or triggered a Lucy Assist, all the proceeding actions are tracked.

Lucy Assist Report Elements:

  • company_name - Name of the company on behalf of which the user is asking a question
  • first_name - The user’s first name (option to use non-PII Key)
  • last_name - The user’s last name (option to use non-PII Key)
  • email - The user’s email address (option to use non-PII Key)
  • question_id - The unique ID number assigned to every question
  • question - The question asked by the user
  • event- The event that occurred when asked “Did you find what you were looking for?”
    • Event Actions:
      • Auto-fire message - This occurs when there are no relevant results
      • Lucy Assist Manual - No - This occurs when a user replies with a “No”
      • Lucy Assist Manual - No - Note - This occurs when a user replies with a “No” and a message for additional support
      • Lucy Assist Manual - No Starmind No - *This is only for clients with a Starmind Integration and occurs when a user says “No” they didn’t find what they were looking for and “No” to posting their question to Starmind
      • Lucy Assist Manual - No Starmind Yes - *This is only for clients with a Starmind Integration and occurs when a user says  “No” they didn’t find what they were looking for and “Yes” to posting their question to Starmind
      • Lucy Assist Manual - Yes - This occurs when a user replies with a “Yes”
      • Lucy Assist Manual - Yes - Time Saved - This occurs when a user replies with a “Yes” and how much time they saved 
  • data - Any notes that are relevant to the question asked by the user
  • date - The date on which the question was asked


USERS REPORT

This is an export of all users within your instance along with their associate names, email, and active status.

User Report Elements: 

  • user_id - The unique ID assigned to each user
  • first_name - The user’s first name (option to use non-PII Key)
  • last_name - The user’s last name (option to use non-PII Key)
  • email - The user’s email address (option to use non-PII Key)
  • group - If applicable, the group/sector of the user
  • is_active - Whether or not the user is an active user
  • last_log_in - The date of the user’s last login
  • sources - The sources and folders the user has access to


CMS REPORT

This report contains all the activity across the Capabilities, Community and Quick Links pages. The activity is gathered when users browse to these pages, as well as what links or portals they clicked on.

CMS Report Elements:

  • company_name - Name of the company on behalf of which the user is asking a question
  • first_name - The user’s first name (option to use non-PII Key)
  • last_name - The user’s last name (option to use non-PII Key)
  • email - The user’s email address (option to use non-PII Key)
  • events - How the user has interacted with the CMS
    • Event Actions:
      • Navigate to Quick Links
      • Quick Link Alternate Link Clicked
      • Quick Link Clicked
      • User Clicked into Capability Card via the homepage
      • User Clicked into Community Card via the homepage
      • User navigated to Capabilities page
      • User navigated to CMS Article
      • User navigated into CMS Article via the homepage
      • User navigated into CMS Article via the nav bar
      • User navigated to Capabilities page
      • User navigated to Community page
      • User navigated within Community page - Card clicked
      • User navigated within Community page - Card clicked
      • CMS Section Subscribed
      • CMS Section Unsubscribed
  • type - The flexible CMS type or name
  • section - the CMS section the article is assigned
  • title - the title of the CMS article
  • referrer - where the user came from (in browse view)
  • homepage - whether the user came from the homepage
  • date - The date on which the actions were taken
  • attributes - If applicable, assigned attributes 


FILTER REPORT

All user data related to the filtering of a question in the Lucy platform.

Filter Report Elements: 

  • company_name - Name of the company on behalf of which the user is asking a question
  • first_name - The user’s first name (option to use non-PII Key)
  • last_name - The user’s last name (option to use non-PII Key)
  • question asked - The question asked by the user
  • date - The date on which the filter was applied
  • source filter - The source filter that was applied
  • timeframe filter - The timeframe filter that was applied
  • filetype filter - The filetype filter that was applied
  • content filter - The content filter that was applied


TRAINING REPORT

All user data related to the feedback provided to answers presented within the Lucy platform. The report also includes the total amount of questions asked to allow for a quick comparison of feedback as a percentage of total questions asked. This report has 5 tabs which are: 

  1. Training Report
  2. All Questions
  3. Summary
  4. Training Summary
  5. Questions Summary

Training Report Elements:

  • Company - Name of the company on behalf of which the user is asking a question
  • Email - The user’s email address (option to use non-PII Key)
  • Question - The question asked by the user
  • Answer- The answer id which was trained
  • Training- If the user has trained a specific answer
    • Event Actions:
      • Positive - The user clicked thumbs up on the answer
      • Negative - The user clicked thumbs down on the answer
  • Position - The position (1 - 10) the answer was presented in
  • Date - The date and time at which the question was asked

All Questions Report Elements:

  • Company - Name of the company on behalf of which the user is asking a question
  • Email - The user’s email address (option to use non-PII Key)
  • Question - The question asked by the user
  • Date - The date and time at which the question was asked

Summary Elements:

  • Span of Days - Days in which the training occurred within the instance
  • Count of Positive Trained - Total number of positive (thumbs up) trainings applied
  • Count of Negative Trained - Total number of negative (thumbs down) trainings applied
  • Total Training - Total count of training events
  • Trainings per Question - Total answers trained divided by total questions asked
  • Positive Trainings Per Question - Total answers trained positively (thumbs up) divided by total questions asked
  • Positive Training Ratio - Positive training (thumbs up) over total training events (thumbs up & thumbs down)
  • Trainings per Day - Total training events (thumbs up & thumbs down) divided by span of days in report
  • Questions per Day - Total question asked divided by span of days in report 

Training Summary Elements: Pivot and graph of training events (positive & negative) applied per day 

Questions Summary Elements: Pivot and graph of questions asked per day


INDEXING  REPORT

Report of all file status for all sources configured within your instance.

Indexing Report Elements:

  • File Name - Shows the file name for indexed documents
  • Size in Bytes - The size of the file ingested in Bytes
  • Pretty Size - The size of the file ingested in MB
  • Page Count - Page count for documents
  • File URL - URL for the connected file
  • Status - Shows the status of the file
    • Complete
    • New
    • Ignored
    • Attention
  • Source ID - Unique ID number for file in the system
  • Source Name - Name of the source the document is located in
  • Source URL - The URL of the content source


Was this article helpful?