探索 Metavision® 智能套件
荣获五十余项大奖的事件视觉软件工具包。
免费下载,免费制作,免费销售
领先的基于事件的机器学习工具包
Prophesee花了6年多的时间来完善这一庞大的预构建管道、广泛的数据集、代码示例和GUI工具等,确保您可以在几分钟内得到结果。
免费下载、免费搭建、免费商业化授权
Prophesee的软件工具包可以在PC端Windows和Ubuntu系统中完全免费使用,没有任何时间限制。它包括一个商业级的许可证,使您能够利用Prophesee 基于事件的元视觉传感技术,可用于设计和销售自己基于元视觉智能的产品。
软件
检测推理
一套专门的工具为您提供了使用事件执行深度神经网络(DNN)所需的一切,以便释放基于事件的机器学习的潜力。您可以充分利用我们用pytorch编写的预训练的汽车模型,使用我们的c++管道进行实时检测和跟踪实验。使用我们功能全面的python库,来设计您自己的网络。
实时检测和跟踪 @100Hz
检测训练
您可以随时使用我们的训练框架训练您自己的物体检测应用。您可以实验多个预先构建的基于事件的张量表示,训练适合事件的数据的网络。
自动生成的 HDF5 数据集
功能全面的训练工具箱,包括量身定做的预处理、DataLoader、NN架构、可视化工具等。
视频到事件
视频到事件管道连接基于帧和基于事件的世界。生成合成数据以扩充数据集,并部分重复使用现有的资料。
无缝集成与GPU兼容的事件模拟器
事件到视频
使用该神经网络根据事件构建灰度图像。对于基于帧方案已解决的任务,可以借此来做事件数据的自动注释,以节省资源。
充分利用现有的基于帧的神经网络 来获得标签
经检测KPI
用我们符合最新COCO API标准的物体检测KPI工具包来评估您的检测性能。
包括 mAP, mAR 和它们的变体
光流推断
利用我们预先训练的流模型,从基于事件的数据中预测光流;利用我们预先训练的光流模型、定制的数据加载器以及损失函数和可视化工具集合来建立你的光流预测管道。
轻量级模型
光流训练
没有真实值?充分利用我们的自监督架构。用我们定制的FlowNet训练框架来训练您的光流应用,为您基于事件的数据量身定制4个预先构建的流网络。
手势分类推理
用我们预先训练好的模型在事件相机或事件数据中运行一个实时的剪刀石头布游戏。
手势分类训练
用这个量身定做的基于事件的训练管道来训练您的物体分类器。
振动监测
通过跟踪场景中每个像素的时序信号,以像素级的精度连续、远程地监测振动频率。每个事件,像素坐标、变化的极性和准确的时间戳都被记录下来,从而提供对振动模式的全面、连续的理解。
1 个像素 的精度
飞溅物监测
追踪具有溅射状运动的小颗粒。事件的视觉传感器由于具有高时间分辨率和动态范围,可以在最苛刻的环境中追踪小颗粒。
>同时对所有颗粒进行XYT追踪
高速计数
以高速对物体进行计数,精确度高,产生的数据少,没有任何运动模糊。物体在经过视野时被计数,当有物体运动时,相应像素才会被触发。
精度:>99.5% @1,000 个物体/秒。
粒径监测
控制、计数和测量在通道或传送带上以极高速度移动的物体的尺寸,以便在您的生产线上获得即时的质量统计数据,以控制生产过程。
99.9% 的计数精度
物体追踪
追踪视野中的移动物体,利用基于事件的传感器具备的低数据率和稀疏化特性,在低算力条件下跟踪物体。
原生分割: 只分析运动,忽略静态背景
标定
在现实环境中部署应用程序,并控制基于事件的系统的所有光学参数。我们提供一整套预置的工具来标定您的相机和调整焦点。根据您的具体需要进行扩展,并使用我们的算法模块连接到标准标定程序。
自动相机内参标定
边缘跟踪
为您的AR/VR应用追踪3D边缘和/或基准点。受益于事件的高时间分辨率,可以提高您的边缘跟踪应用的准确性和鲁棒性。
三维物体实时跟踪
光流
通过事件,重现审视这一计算机视觉基础构建模块。颠覆传统的逐帧分析方法,利用连续的逐个像素的跟踪,更有效地理解运动。
与传统的基于图像的方法相比,功耗降低17倍
只在移动物体上采集特征点。
XYT 视觉化
通过我们的XYT查看器将您的数据可视化,探索时间-空间连续性对您应用的效用。
放大时间,了解场景中的运动
数据率可视化
了解事件随时间产生的过程,利用python的功能实现数据的可视化并生成图表。
与数据文件和事件相机的接口
超慢动作
放慢时间,慢到时间分辨率相当于每秒超过20万帧,而生成的数据比传统方法少了几个数量级。帮助了解隐藏在超高速的和极短暂的事件中的精细的运动动态。
工具
无论你是否拥有EVK,Metavision Studio都可以作为您完美的启动工具。
其具有图形用户界面,任何人都可通过与PROPHESEE兼容的基于事件的视觉系统进行可视化和记录数据流。
它还能够使您浏览其所提供的事件数据集,以加深对基于事件的视觉的理解。
可视化、信息流、调整事件
全面的图形用户界面
主要特点
- 来自与Prophesee兼容的基于事件的视觉系统的数据可视化
- 数据可视化控制(累积时间,fps)
- 数据记录
- 回放记录数据
- 导出为AVI视频
- 控制,保存和加载传感器设置
其具有图形用户界面,任何人都可通过与PROPHESEE兼容的基于事件的视觉系统进行可视化和记录数据流。
它还能够使您浏览其所提供的事件数据集,以加深对基于事件的视觉的理解。
可视化、信息流、调整事件
全面的图形用户界面
主要特点
- 来自与Prophesee兼容的基于事件的视觉系统的数据可视化
- 数据可视化控制(累积时间,fps)
- 数据记录
- 回放记录数据
- 导出为AVI视频
- 控制,保存和加载传感器设置
Metavision SDK is the largest set of Event-Based Vision algorithms accessible to date. High-performance algorithms are available via APIs, ready to go to production with Event-Based Vision applications.
The provided algorithms are available as C++ APIs for highly efficient applications and runtime execution or Python APIs giving access to the C++ algorithms, Machine Learning pipelines and more.
Develop high-performance Event-Based Vision solutions
25 c++ code samples, 44 python classes 51 algorithms, 26 python samples, 16 tutorials and 11 ready-to-use applications
Main features
- Extensive documentation / code examples / training & learning material available
- Runs natively on Linux and Windows
- Compatible with Prophesee vision systems and « Powered by Prophesee » partners products
- A complete API that allows you to explore the full potential of Event-Based Vision in just a few lines of code
/********************************************************************************************************************** * Copyright (c) Prophesee S.A. * * * * Licensed under the Apache License, Version 2.0 (the "License"); * * you may not use this file except in compliance with the License. * * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed * * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * * See the License for the specific language governing permissions and limitations under the License. * **********************************************************************************************************************/ // This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event // counter and displayer by introducing some basic concepts of the Metavision SDK. #include <metavision/sdk/driver/camera.h> #include <metavision/sdk/base/events/event_cd.h> #include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h> #include <metavision/sdk/ui/utils/window.h> #include <metavision/sdk/ui/utils/event_loop.h> // this class will be used to analyze the events class EventAnalyzer { public: // class variables to store global information int global_counter = 0; // this will track how many events we processed Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed // this function will be associated to the camera callback // it is used to compute statistics on the received events void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) { std::cout << "----- New callback! -----" << std::endl; // time analysis // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and // the last event will have the highest timestamp Metavision::timestamp min_t = begin->t; // get the timestamp of the first event of this callback Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp // counting analysis int counter = 0; for (const Metavision::EventCD *ev = begin; ev != end; ++ev) { ++counter; // increasing local counter } global_counter += counter; // increase global counter // report std::cout << "There were " << counter << " events in this callback" << std::endl; std::cout << "There were " << global_counter << " total events up to now." << std::endl; std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds." << std::endl; std::cout << "----- End of the callback! -----" << std::endl; } }; // main loop int main(int argc, char *argv[]) { Metavision::Camera cam; // create the camera EventAnalyzer event_analyzer; // create the event analyzer if (argc >= 2) { // if we passed a file path, open it cam = Metavision::Camera::from_file(argv[1]); } else { // open the first available camera cam = Metavision::Camera::from_first_available(); } // to analyze the events, we add a callback that will be called periodically to give access to the latest events cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) { event_analyzer.analyze_events(ev_begin, ev_end); }); // to visualize the events, we will need to build frames and render them. // building frame will be done with a frame generator that will accumulate the events over time. // we need to provide it the camera resolution that we can retrieve from the camera instance int camera_width = cam.geometry().width(); int camera_height = cam.geometry().height(); // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps) const std::uint32_t acc = 20000; double fps = 50; // now we can create our frame generator using previous variables auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps); // we add the callback that will pass the events to the frame generator cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) { frame_gen.process_events(begin, end); }); // to render the frames, we create a window using the Window class of the UI module Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height, Metavision::BaseWindow::RenderMode::BGR); // we set a callback on the windows to close it when the Escape or Q key is pressed window.set_keyboard_callback( [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) { if (action == Metavision::UIAction::RELEASE && (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) { window.set_close_flag(); } }); // we set a callback on the frame generator so that it calls the window object to display the generated frames frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); }); // start the camera cam.start(); // keep running until the camera is off, the recording is finished or the escape key was pressed while (cam.is_running() && !window.should_close()) { // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core // and we push them into the window where the callback on the escape key will ask the windows to close static constexpr std::int64_t kSleepPeriodMs = 20; Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs); } // the recording is finished or the user wants to quit, stop the camera. cam.stop(); // print the global statistics const double length_in_seconds = event_analyzer.global_max_t / 1000000.0; std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl; std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl; if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short std::cout << "There were " << event_analyzer.global_counter / length_in_seconds << " events per second on average." << std::endl; } }
Metavision SDK是迄今为止最大的基于事件的视觉算法集。其高性能算法可通过api获得,辅以基于事件的视觉应用程序,已可投入生产。
所提供的算法可以作为C++ api提供高效的应用程序和运行执行,也可以作为Python api获取对c++算法、机器学习管道等的访问。
开发高性能基于事件的视觉解决方案
32 个C++代码示例, 112 个Python类63 个算法, 47 个Python示例, 24个教程和17个现成的应用程序
主要特点
- 大量的文档/代码示例/培训和学习材料可用
- 本机在Linux和Windows上运行
- 兼容Prophesee视觉系统和«由prophecy提供支持的»合作伙伴产品
- 一个完整的API,允许您仅用短短几行代码即可探索基于事件的视觉的巨大潜力。
/********************************************************************************************************************** * Copyright (c) Prophesee S.A. * * * * Licensed under the Apache License, Version 2.0 (the "License"); * * you may not use this file except in compliance with the License. * * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed * * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * * See the License for the specific language governing permissions and limitations under the License. * **********************************************************************************************************************/ // This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event // counter and displayer by introducing some basic concepts of the Metavision SDK. #include <metavision/sdk/driver/camera.h> #include <metavision/sdk/base/events/event_cd.h> #include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h> #include <metavision/sdk/ui/utils/window.h> #include <metavision/sdk/ui/utils/event_loop.h> // this class will be used to analyze the events class EventAnalyzer { public: // class variables to store global information int global_counter = 0; // this will track how many events we processed Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed // this function will be associated to the camera callback // it is used to compute statistics on the received events void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) { std::cout << "----- New callback! -----" << std::endl; // time analysis // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and // the last event will have the highest timestamp Metavision::timestamp min_t = begin->t; // get the timestamp of the first event of this callback Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp // counting analysis int counter = 0; for (const Metavision::EventCD *ev = begin; ev != end; ++ev) { ++counter; // increasing local counter } global_counter += counter; // increase global counter // report std::cout << "There were " << counter << " events in this callback" << std::endl; std::cout << "There were " << global_counter << " total events up to now." << std::endl; std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds." << std::endl; std::cout << "----- End of the callback! -----" << std::endl; } }; // main loop int main(int argc, char *argv[]) { Metavision::Camera cam; // create the camera EventAnalyzer event_analyzer; // create the event analyzer if (argc >= 2) { // if we passed a file path, open it cam = Metavision::Camera::from_file(argv[1]); } else { // open the first available camera cam = Metavision::Camera::from_first_available(); } // to analyze the events, we add a callback that will be called periodically to give access to the latest events cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) { event_analyzer.analyze_events(ev_begin, ev_end); }); // to visualize the events, we will need to build frames and render them. // building frame will be done with a frame generator that will accumulate the events over time. // we need to provide it the camera resolution that we can retrieve from the camera instance int camera_width = cam.geometry().width(); int camera_height = cam.geometry().height(); // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps) const std::uint32_t acc = 20000; double fps = 50; // now we can create our frame generator using previous variables auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps); // we add the callback that will pass the events to the frame generator cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) { frame_gen.process_events(begin, end); }); // to render the frames, we create a window using the Window class of the UI module Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height, Metavision::BaseWindow::RenderMode::BGR); // we set a callback on the windows to close it when the Escape or Q key is pressed window.set_keyboard_callback( [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) { if (action == Metavision::UIAction::RELEASE && (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) { window.set_close_flag(); } }); // we set a callback on the frame generator so that it calls the window object to display the generated frames frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); }); // start the camera cam.start(); // keep running until the camera is off, the recording is finished or the escape key was pressed while (cam.is_running() && !window.should_close()) { // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core // and we push them into the window where the callback on the escape key will ask the windows to close static constexpr std::int64_t kSleepPeriodMs = 20; Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs); } // the recording is finished or the user wants to quit, stop the camera. cam.stop(); // print the global statistics const double length_in_seconds = event_analyzer.global_max_t / 1000000.0; std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl; std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl; if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short std::cout << "There were " << event_analyzer.global_counter / length_in_seconds << " events per second on average." << std::endl; } }
硬件
购买兼容的硬件
构建兼容硬件
通过广泛的合作计划,Prophesee 帮助视觉设备制造商构建其基于事件的视觉产品。
获取源代码,开发您自己的硬件,或将您的视觉系统与Metavision Intelligence集成。