博文
端对端平行无人矿山系统及其关键技术
|
端对端平行无人矿山系统及其关键技术
杨超，高玉 ，艾云峰 ，田滨 ，陈龙 ，王健 ，王飞跃
【摘 要】针对矿山无人化的迫切需求与研究现状，提出了基于 ACP 方法的端对端平行无人矿山系统解决方案。 首先分析了无人矿山运输系统的研究现状；设计了平行无人矿山的基本框架；提出了平行无人矿山运输系统的主 要模块，包括：无人矿山管控中心、无人矿卡运输平台、半自主挖掘/铲运平台和远程接管平台 4 部分；并探讨了实现平行无人矿山系统的 7 个关键技术，即虚拟端平行矿山构建、矿山环境感知与定位、矿山设备的单机无人控 制、多设备之间的协同作业、平行矿山管控、平行接管决策以及矿山网络通信。平行无人矿山系统的实施与部署将有效提高矿山企业运作效率及安全生产水平，推进绿色矿山建设，对实现可持续发展具有重要意义。
【关键词】平行无人矿山；管控中心；接管决策；协同作业；无人矿卡；半自主挖掘/铲运
引用格式 杨超, 高玉, 艾云峰, 田滨,陈龙, 王健, 王飞跃. 端对端平行无人矿山系统及其关键技术. 智能科学与技术学报[J], 2019, 1(3): 228-240
End-to-end parallel autonomous mining systems and key technologies
YANG Chao, GAO Yu, AI Yunfeng, TIAN Bin, CHEN Long, WANG Jian, WANG Fei-Yue
Abstract In view of the current research situation and the urgent demand for unmanned mines, the end-to-end parallel autonomous mining solution was proposed based on the ACP method in this paper. The research status of unmanned mining transportation system was analyzed. The basic framework of parallel autonomous mining was designed. The par- allel autonomous mining are mainly composed of four parts: the management and control center for autonomous mining, the autonomous transportation platform of truck, the autonomous mining/shovel platform and the remote take-over plat- form. The 7 key technologies for realizing parallel autonomous mining system were discussed, namely, the virtual paral- lel mine construction, environment perception and localization, unmanned control for single equipment, collaborative work between multi-equipment, management and control for all the equipment, decision for parallel taking over, and mine network communication. The implementation and deployment of parallel autonomous mining systems will effec- tively improve the operational efficiency and the level of safe production of mining enterprises, and will promote the construction of green mines, which is of great significance for achieving sustainable development.
Key words parallel autonomous mining, management and control center, take-over decision, collaborative work, au- tonomous truck, autonomous mining/shovel
Citation YANG Chao. End-to-end parallel autonomous mining systems and key technologies. Chinese Journal of Intelligent Science and Technology[J], 2019, 1(3): 228-240
1 引言
作为汽车产业变革的发力点，自动驾驶技术不 仅是国内外众多传统车企转战的重点，也是各大互联网巨头关注的焦点之一。然而，无人车在公共道路开放场景的商业落地面临诸多瓶颈。复杂的交通 标志、道路参与者难以预知的行为以及性能不稳定 的硬件设备等都是无人车走向城市道路的阻碍因 素。因此，在限定场景、中低速工况下的专用车将 是无人驾驶技术最先落地并商业化的应用场合。矿 山系统作为相对封闭的场景，已成为众多无人驾驶 相关企业关注的重要领域之一。
矿山系统是一个典型的复杂系统，它包括生产（开采）子系统、调度子系统、供水供电子系统、 运输子系统、设备及人员管理子系统、安全子系统 等各个方面。要实现矿山系统的无人化，生产、运 输、调度、安全等子系统是关键[1]。
然而受矿山地理位置偏僻、生活环境较为封 闭、生产条件较艰苦等的影响，随着社会的进步及 人们生活品质的提高，露天矿招工困难的问题日益 凸显，年轻司机从业意愿低，导致熟练工人的招聘 难度逐年加大。同时露天煤矿在矿区运输问题上的 其他痛点也日益突出，一方面，人工驾驶卡车带来 的安全生产风险难以从本质上消除，运输环节安全 风险集中性高；另一方面，矿区运输成本逐年升高， 对矿山生产的影响持续加大。市场需求是制造商开 发和生产的动力，在无人矿山领域，工程巨头美国 卡特彼勒公司和日本小松公司走在前面，已经实现 了无人卡车在矿山区域的自动化商业运行。白俄罗 斯的别拉斯、瑞典的山特维克和沃尔沃等也推出了无人矿卡产品。目前使用自动化卡车的矿山所占比例还很小，当前模式是工程机械制造商与矿业公司 合作，并为后者提供解决方案，最主要的是卡特彼勒公司与必拓公司、小松公司和力拓集团。
在矿卡需求方面，目前全球矿卡每年销售总量（不包括管理系统）在 1 000 台左右，总采购成本大约为 20 亿美元。预计到 2020 年，澳洲、欧洲和美洲的主要矿业公司中 50%将使用无人化产品，部分 矿山已经决定全部采用无人矿卡。但目前无人矿卡 的占比仅不到 5%，主要由卡特彼勒公司和小松公司两家矿卡生产商集中供应。
对于矿山企业，实现矿山无人化具备以下优势：增加效率和生产率；减少司机费用开销，降低劳动力成本；增强矿山的安全性，减少工作意外； 降低运行成本，减少排放；受极端环境的影响减少； 便于对车辆统一管理。
综上所述，无人矿山具备巨大的市场潜力， 同时也能够给用户（矿山开采企业）带来成本、 安全、效率、环保等多方面的利好。相应地，也会给矿山车辆及设备制造商带来丰厚的盈利，具体包括：无人矿山专用卡车销售溢价；在常规矿卡基础上的无人矿卡改装；提供无人矿卡整套软件技术解决方案；提供矿山车辆远程监控与调度 管理系统。除上述直接的盈利增加外，无人矿卡的采用还会促进金融保险、租赁、维修等后市场 收入的进一步提高。
2 无人矿山研究现状
早在 20 世纪 90 年代，数字矿山的概念就已经出现。已有百年历史的采矿业在生产管理方式上不断向数字信息化发展，这种发展的目的是在矿山作业环境安全可靠的基础上实现矿山发展的可持续性。矿山的数字信息化安全作业主要是运用远距 离测量控制以及动态仿真等技术，实现对大量地质 勘测图和矿山开采图以及相关数据的科学管理和 记录，并且基于这些数据实现三维可视的目的，为 科学开采和管理矿山提供可靠依据[2]。
为实现矿山无人化运输，小松公司推出了自动 运输系统（AHS）用于矿山车队的运营和管理[3]。 此系统下的每辆无人卡车都配有车辆控制器、高精 度定位系统、障碍物检测系统和无线网络系统，由 车队监控中心为每辆车指定运输目的地，车辆根据 接收到的指令信息实现装载、运输和卸载循环的自 动运行。在安全方面，无人卡车能够自动检测人员和车辆，避免发生碰撞。2018 年 10 月，小松公司 宣布，其 AHS 下的卡车（包含有人卡车和无人卡 车）已实现 20 亿吨的矿石运输[4]。
工程机械巨头卡特彼勒公司亦是矿卡自动化领域的活跃者，它在 2004 年的矿业博览会上展出了矿卡自动控制系统（MineGEM）[5]，可以使操作人员在危险作业区域之外或者远程控制室操作采矿设备。之后推出了综合性的采矿作业与设备管理 系统——卡特彼勒公司矿山之星（Cat MineStar）， 该 系 统 由 许 多 性 能 套 件 组 成 ， 目 前 包 括 ： 车队（Fleet）、地形（Terrain）、检测（Detect）与指令（Command）套件。性能套件为全集成式，所有信 息可被整个系统共享，用于优化生产率、增强安全 性，提升机器的利用率及工作时间。卡特彼勒公司宣 称其综合性采矿作业与设备管理系统已实现 10 亿吨 矿石的开采和运输。
山特维克也是世界上较早开发自动化采矿技术的公司，但其自动化铲运机和卡车主要在井下矿山应用，并保持着 20 多年零工伤事故的纪录。为证明其实力，山特维克在地上搭建的人工模拟巷道中对其新一代自动化产品进行了自我挑战。
2016 年秋天，沃尔沃的无人驾驶卡车在瑞典 北部的矿区 Boliden 煤矿地下 1 320 m 进行了测试， 并真正投入使用，验证了自动驾驶技术在严苛环境 中的安全性以及在生产运输环节中的潜在价值。
我国矿山的信息化经过 20 多年的发展，已经取得了显著的成效，但仍存在诸多问题，主要体现在以下 5 个方面：缺乏信息化标准，导致种类繁多的系统差异；缺乏多学科交叉应用，无法形成一个能搭载各学科知识应用的开放性公共平台；软件投入及开发不足；矿山子系统之间的数据共享与融合分析无法顺利开展，产生大量“信息孤岛”；管理系统复杂、效率低下、维护成本过高等[6]。
综上所述，国外企业像卡特彼勒公司、小松公司、日立、沃尔沃等均已实现了不同程度的矿山无人化运作，我国在无人矿山的进程方面也在不断加快步伐，整体上已经具备在单台矿用卡车上做无人驾驶的能力，但在设备间的协同作业以及集群化运营方面，还需要更多的研究，以实现对无人矿山 安全和效率的提升。更重要的是，我国无人化矿山技术缺乏足够的理论支撑，在知识产权等方面 受限于国外。
为此，通过建立一个与现实矿山系统相对应的 动态人工系统（即平行矿山系统），利用这个人工系统对实际复杂系统进行实时、动态、在线的模拟 计算，并通过对人工系统演化及预测的研究，有助于实现对现实矿山系统的精准高效预测与管理。人工生命、人工社会以及计算实验等研究方法的提 出，更为这一思想的实施奠定了技术基础。本文利 用现代信息技术（如物联网技术、云计算技术、无 线通信技术等），采集矿山的各个关键系统的实时 数据，并将其传输至管理中心，管理中心利用以上 数据构建一个与现实系统相对应的人工系统，并通过对人工系统的演化实现对现实系统的预测、评 估、优化，从而实现对矿山的优化管理。
3 平行无人矿山整体框架
3.1 ACP 方法简介
中 国 科 学 院 自 动 化 研 究 所 王 飞 跃 研 究 员 于 2004 年提出了平行系统的思想，即通过构建与真实 系统虚实互动的人工系统，来控制、管理、优化和 引导复杂系统[7]。平行系统的核心技术方法是 ACP 方法，其中 A 是人工社会，即与真实系统运行机理 一致的人工系统模型；C 是计算实验，即基于人工 系统，利用当前高效、稳健的人工智能、机器学习等方法来对构建的不同复杂问题进行试错求解和 分析；P 是平行执行，即用人工系统来指导真实系 统，同时真实系统也可提升人工系统[8-9]。
平行系统中 ACP 方法的基本框架如图 1 所示。 在人工系统的构建过程中，需保证虚拟系统与真实 系统的一致性，只有在足够准确的模型基础上获得 虚拟控制和决策，才能对真实的系统起到指导作 用。在具体的实施过程中，主要通过描述学习方式 来描绘和构建系统。同时，模型的精度是可以在平 行执行的过程中实时提升的[10]。
计算实验是在虚拟系统和真实系统两个维度中同时进行的，而且有越来越智能的趋势。具体来说，在实际系统中，计算实验主要对过去和当前的数据信息进行处理，获得能够指导实际系统运行的知识；而在虚拟系统中，计算实验则主要体现为通过预测学习和引导学习来获取未来的或者未经历 过的知识，从而使得平行系统更加完备[11-12]。
平行执行则无时无刻不在平行系统中体现，包括虚拟系统和真实系统之间的数据流、控制流、模型流等。反馈是虚拟系统流向真实系统的描述，通过大量的人工智能、机器学习等算法来获得知识，用以引导真实系统；提升是真实系统流向虚拟系统的描绘，实际系统的知识能够更好地反馈当前整个系统的走向和趋势，从而对虚拟系统的模型进行引导和修改[13-15]。
3.2 平行无人矿山的集群化运营分析
基于 ACP 方法的平行无人矿山框架如图 2 所示， 它是平行系统在矿山领域的一个切实应用。平行无人 矿山中主要包含 6 个重要的部件（地点），分别是采 矿点（A）、卸矿点（B）、传输带卸矿点（C）、货运 火车站（D）、管控中心（E）以及虚拟平行矿山系统（F）。其中 A、B、C 和 D 可以看作真实系统的组成 部分，而 E 和 F 可以看作虚拟系统的组成部分。
在平行无人矿山中，主要研究和解决 4 个方面 的问题。第一个是在 A 点，关于挖掘机的半自主挖 掘作业，可通过挖卡协同作业提升挖掘效率。第二个是 A 点到 B 点的无人矿卡运输，通过在大型矿卡中加装传感器和智能感知、决策和控制算法，实现 无人矿卡的自动驾驶。第三个是 C 点到 D 点的无人 重型卡车运输，与无人矿卡相同，无人重型卡车的 运行是全自动的，可实现自动装载、运输、自动卸载（包括环境的感知、路径的规划、速度的规划、 避障）等功能。第四个是平行无人矿山管控，所有 设备的自动运行都受到管控中心的监管、控制与调 度，同时各设备之间互相通信，实现协调控制。这 四大功能的协调运行，能够实现整个矿山作业的无 人化，从而提升整个系统的安全和效率。
3.3 平行无人矿山的优势
我国在矿山智能化和无人化的建设方面仍然存在如前文所述的一些不足，面对如此复杂、巨大的系统的控制和管理，平行理论和 ACP 方法将为矿山智能化和无人化提供良好的解决方案[16]。利用平行智能理论和 ACP 方法建立的平行无人矿山系统是一个高度网联的智能化系统，其主要优点包括：大幅度提升人员和设备的安全程度；降低人力成本、燃油成本、设备与路面维护成本；提升矿山系统的智能化水平，减少操作人员，提高风险管控能力；优化矿山系统的集群管理，提高调度及协同效率，实现产量的增加；减少人为失误和故意损坏导致的损失。
平行矿山系统中，每一个子系统都会建立一个 与之对应的平行子系统，将现实空间的物理系统映入虚拟空间建立平行系统，通过数字化计算联网、 调度以及管理的最优策略，再投入实施，可以解决 各子系统网联化程度低、调度难、管理混乱等问题。
将平行理论应用到智能矿山系统，以数据为驱动，构建与实际矿山系统互动的系列虚拟矿山组织 及机构，形成平行无人矿山系统，利用计算实验对 各类复杂的矿山调度、协同、管理、控制、通信等问题进行分析和评估，目标是变革智能化背景下矿山系统的生产模式，提升矿山子系统间的协同水平 与智能化程度。
利用 ACP 方法建立的平行矿山系统，其基本 思想是分别建立矿山各子系统的平行子系统，如平行矿卡、平行挖掘机、平行管理、平行协同通信等子系统，通过子系统的自身局部优化和整体系统优 化相结合的迭代优化方式，并行完善每一个平行子系统的功能，进而建立最优的全局系统。
平行矿山系统将一个管理难、控制难、预测难 的矿山复杂巨系统以功能为单位拆分成多个平行 子系统，可以灵活优化矿山系统的各个组成部分；同时，还可以通过计算实验和平行执行的方式将在 虚拟世界中探索获得的预测与管理策略作用于现 实矿山系统，使现实矿山系统能够应对未知的工作场景和任务。
4 平行无人矿山系统及功能设计
平行无人矿山由四部分组成：平行无人矿山管 控中心、无人矿卡运输平台、半自主挖掘/铲运平台 以及远程接管平台。平行无人矿山管控中心负责对 无人矿山进行全面实时监控与综合调度，通过所建立的虚拟矿山与实际矿山的实时互动与双向优化， 保障实际矿山高效运营与安全作业。无人矿卡运输平台以及半自主挖掘/铲运平台搭载多种传感设备， 实现矿山的半自主挖掘、装载、无人化运输、卸载 一体化作业。远程接管平台负责监控矿山设备的生 产运营，在无人矿卡运输平台或半自主挖掘/铲运平台主动提出接管请求时，或在特殊紧急情况下，经由平行无人矿山管控中心的综合决策下达接管指令，由远程接管平台实现对多台设备的干预与接 管。如图 3 所示。
4.1 平行无人矿山管控中心
平行无人矿山管控中心是矿山体系的神经中枢，基于 ACP 方法构建契合矿山特色的管理与控制中心，对矿山进行全天候、全方位的监测与管理，实现生产方案优化、资源调度、生产设备监测与维护、人力资源管理等功能，保障无人矿山的安全生产、高效运营。
平行矿山管控中心包括虚拟矿山系统与矿山 计算实验系统两大功能模块。通过对实际矿山系统 进行动态实时数据采集与三维可视化仿真，建立虚拟矿山系统，与实际矿山虚实对应、动态演化、平行执行，实现对实际矿山的预估与引导，从而优化 无人矿山系统的管理与控制。平行无人矿山管控中 心平台包括仿真设备、工控机、服务器、视频监控 平台、图像拼接器、交换机、异地组网设备等。各 项设备与管控中心的功能相对应，若矿山现场发生 危险情况，或发生与其他系统相关联并且影响其他 系统的正常运行或作业人员安全的情况，管控中心 将做出智能决策，在中心的大屏视频中可实时监测 矿山现场，进行应急指挥与控制。
4.2 无人矿卡运输平台
无人矿卡运输平台主要包括短途运输车辆（无人宽体自卸车）与中长途运输车辆（无人矿用卡车）。所搭载的主要传感器包括激光雷达、毫米波 雷达、惯性导航系统、工业相机、交换机、V2X（车 联网）设备、工控机、无线网络测试仪、车载显示器、远程监控等。无人矿卡由控制中心管理控制， 为每辆车指定运输路线，车辆通过接收无线指令， 以合适的速度按照目标路线运行，根据行驶路线、 自身位置、周围环境等信息自动行驶，完成装载、 运输、卸载的循环。
无人矿卡运输平台包括自动驾驶、模式切换、 状态检测与显示、信息管理与通信和模拟演示等模 块。无人矿卡运输平台的感知子系统通过激光雷 达、毫米波雷达、工业相机感知矿山环境信息，连 同惯性导航系统提供的定位信息，为无人矿卡的决策规划子系统和控制子系统提供决策支持。通信子 系统通过与管控中心、远程接管平台的无线通信， 实现管控中心的远程监控，以及与运输平台和挖掘/ 铲运平台的协同作业。数据管理子系统为无人矿卡 的运行进行数据备份，方便数据回放与研究。人机界面通过车载显示器呈现无人矿卡的行驶数据与 车辆状态，提供人机接口，确保运营安全。考虑到 矿山作业安全性的特殊需求，无人矿卡运输平台还 包括状态检测子系统与模式切换子系统。状态检测 子系统检测各子系统的健康状态，模式切换子系统 综合状态检测子系统、管控中心、远程接管平台的 数据以及命令进行决策，在人工模式与自动模式间选择切换。
4.3 半自主挖掘/铲运平台
半自主挖掘/铲运平台包括挖掘机与铲运机等， 通过搭载的位移传感器、压力传感器、激光雷达、惯性导航系统、摄像头、毫米波雷达等设备，可实现在采矿点的半自主挖掘与铲运，同无人矿卡运输 平台协同作业。
在挖掘机与矿卡协同运作中，首先根据挖掘机所在位置实现对作业区域内矿卡行驶路径的自动 规划，由挖掘机引导矿卡行驶至正确的位置进行装 载；挖掘平台根据搭载的各类传感器信息实现对目 标矿物的自动挖掘以及自动卸料；在整个挖掘过程 中，位于管控中心的挖掘机手可根据 360 度全景影 像、挖掘位姿影像等数字化辅助功能对自动挖掘过 程进行全程监控，并在必要时进行人工干预，既保 证了自动挖掘作业的顺利进行，也极大地改善了驾 驶员的工作环境。
而在装载机与矿卡协同作业中，由于装载机移 动更为便捷，需要通过管控中心将矿卡引导至目标位置，由半自主装载机负责矿物的铲装，之后按照 规划路径行驶将矿物运输至矿卡相应位置，最后向矿卡货箱内部投放矿物。
4.4 远程接管平台
矿山作业有较高的安全性要求，对应模式切换功能中的人工模式，远程接管平台可实现对无人矿卡运输平台和半自主挖掘 /铲运平台在特殊异常情况下的远程接管。在故障或者其他紧急情况下，平行无人矿山管控中心向接管平台发送人 工接管请求，切换到人工模式[17]，远程控制无人 矿卡运输平台和半自主挖掘/铲运平台。远程接管 平台根据故障等级的不同实行不同的处理方式， 设计模式切换与接管的多级确认安全策略。一个 远程接管平台可以实现对多辆无人矿卡或者挖掘 机/装载机的接管操控，并且可以通过平行无人矿山管控中心的监测与调度管理，实现远程接管状态下的挖卡协同作业，保障了作业安全与作业质量，同时，远程接管也保障了驾驶员的人身安全， 并提高了工作效率。
5 平行无人矿山的关键技术
平行无人矿山的关键技术主要包括：虚拟端平行矿山构建、矿山环境感知与定位、矿山设备的单机无人控制、多设备之间的协同作业、平行矿山管控、平行接管决策以及矿山网络通信，具体说明如下。
5.1 虚拟端平行矿山构建
构建平行矿山系统的关键是利用信息技术对 矿山系统中各个子系统的关键动态属性进行数据采集；利用现代通信技术实时传输各个采集点的 数据；利用数字建模、虚拟技术、可视化技术等 方法建立与现实相对应的人工系统，将各采集点 的数据实时导入人工系统，使人工系统成为与现 实系统相对应的平行系统；在平行系统中利用各 种智能算法、优化算法对现实系统的作业进行预 测、优化，并将结果反馈给现实系统，从而实现 对现实系统的优化控制。虚拟端平行矿山系统架 构如图 4 所示。
在图 4 中，矿山生产运行过程中的各种数据被 传输至信息处理中心，在信息处理中心通过数据融合、挖掘、可视化处理等手段构建出与实际矿山运行系统平行的各个子系统。通过建立描述矿山、预测矿山和引导矿山 3 个软件定义的虚拟矿山系统， 可以构建一个完整的实际矿山平行系统[18]。这样， 一方面利用实际数据建立虚拟模型，完成对矿山生产作业各方面的并行、在线、实时仿真；另一方面 通过对平行系统的演化及与真实系统的协同交互， 完成对现实系统的预测、评估与优化。
实际工作中，矿山系统复杂多变，难以对其建 立可靠的数学模型，因此矿山生产过程难以精确控 制，存在较大的安全隐患。利用虚拟现实技术实现的多源信息融合、交互式的动态三维可视化场景和实体行为体验，在大量领域均取得了令人瞩目的应用成果。在矿山行业，目前已成功研制出基于虚拟现实技术的安全培训系统、开采模拟系统、车辆 运行模拟系统、灾害过程模拟系统、三维地震可视化分析系统、带式输送机远程监控系统等应用系统[19]。随着技术的不断进步，虚拟现实技术必将在矿井设计、生产管理、风险评价、安全管理与培训、 数据可视化直观展现以及各类综合应用分析等方面 提供更加完善的解决方案。矿山数字信息化平行系 统的建立，可将复杂的实际采矿系统转变为与多个平行子系统相结合的智能系统，对实际状况进行跟踪和测量，通过计算实验和优化，为具体方案的制定提供可靠的科学依据。平行系统的建立可以有效 预估生产过程的发展方向，并为不同情况提供优化 解决方案，大大减少事故发生的可能性，对矿山安 全作业具有重要的指导意义，具有广阔的应用前景。
5.2 矿山环境感知与定位
5.2.1 矿山环境感知
环境感知系统利用传感器（激光雷达、毫米波 雷达、相机、超声波、红外传感器）感知环境信息， 主要完成以下任务：路面及可行驶区域检测、动静 态障碍物检测与跟踪、目标识别以及驾驶员状态监测等。在矿山环境下，路况更为复杂，障碍物种类 繁多，可能的情况包括：空旷区域，路面起伏、颠 簸，路面有积水，路面出现较大障碍物（如石块、树木等），路面为陡峭的坡路等。矿山机械在进行 作业时，采用多传感器融合技术对可行驶区域进行 检测，并在可行驶区域内对障碍物进行检测与跟踪，同时还具有对特定目标进行识别、倒车防碰撞 预警等功能。
检测模块能够检测路面、可行驶区域、障碍物等信息，为安全快速自主导航提供可靠保障。对于路面及可行驶区域检测，可采用雷达、相机等传感器提取道路与可行驶区域，保证矿山机械在 GPS 定位失效的情况下能够沿着可行驶区域安全到达目标点。对于目标检测与跟踪，通过对雷达、相机等传感器数据进行处理，检测出障碍物信息，并对动 态障碍进行跟踪，将此感知信息传输给规划模块， 从而实现矿山机械的自主避障功能。
智能矿山机械的识别系统的识别目标主要包括特种车辆、行人、落石、标识杆、挡墙等。目 标识别方法包括传统模式识别和基于深度学习方法的识别。其中传统模式识别的方法有支持向量机（SVM）、Boosting、最近邻分类器等。基于深度学习的方法有神经网络、卷积神经网络（CNN）、 全卷积神经网络（FCN）、R-CNN、Fast R-CNN、 RPN 等。通过大量的样本特征对网络进行训练， 得到网络参数。
5.2.2 矿山定位
矿用设备的定位技术有：航迹推算定位技术、 无线电定位技术、地图匹配定位技术、视觉定位技术。航迹推算定位技术是指已经确定某一初始时刻矿用设备的位置，结合设备上的传感器推算出车辆的具体位置坐标。无线电定位技术是指利用无线电波的直线恒速传播特性，通过测量固定或运动的物体的位置进行定位的技术。无线电定位有雷达、无线电测向、无线电导航系统和全球 定位系统等。地图匹配定位技术是指预先获得巷 道的数字地图信息，在矿用设备行驶过程中，利 用设备上装载的图像采集系统实时获取设备周围 的环境地图信息，通过和预先存储的地图信息进 行匹配，来获取设备当前的位置信息。视觉定位 技术是指通过电荷耦合器件（CCD）或其他视觉 传感器感知周围环境，矿山设备通过对环境中设定 的 一 些 具 有 明 显 特 征 点 的 路 标 进行 识别和跟 踪，结合特定标识景物的位置信息，进而得到设备的纵向位置信息。
矿山设备工作场景的特殊性（如空旷的野外环境缺少周边固定的特征点和参照物；光线不稳定； 路面泥泞，而且有颠簸；路面的纹理不够，无法使用光流参与局部定位；车辆起点与终点间距离比较 大等）对定位方式提出了更高的要求。具体的定位方式分为地上定位和地下定位。
• 对于地上定位，基于矿山应用场景的特点， 可采用基于差分定位—惯性导航系统（RTK-INS） 和激光定位（LIDAR-SLAM）的组合定位方案，大 部分场景下，RTK-INS 能够完成精度高达厘米级别 的定位要求，但是有些山体和树挡住全球导航卫星 系统（GNSS）或者实时动态（real-time kinematic， RTK）信号的情况下，需要 LIDAR-SLAM 进行辅 助定位。
• 对于地下定位，可采用激光站定位与超宽带（UWB）定位融合的定位方法，定位精度基本可 以达到 10 cm 以内 [20]，也可以采用激光雷达同步定 位与建图（SLAM）的方法进行定位。激光定位系统利用激光测距传感器与目标之间的距离值和伺 服控制机构在空间内所转角度值计算出运动目标 的坐标，实现对地下采矿设备的定位。采用 UWB 定位技术研制的地下采矿设备精确定位系统由两部分组成，一部分是固定在巷道壁上的固定的“锚节点”，这类节点在定位系统中会预先设置好绝对 坐标系下的物理坐标，进行定位计算；另外一部分是系统中的移动节点，安装在移动设备之上，通过 无线通信的各种手段来确定动节点的信息。通过锚 节点与动节点之间的信息交互得出地下采矿设备的精确位置信息。
5.3 矿山设备的单机无人控制
矿山移动设备（如宽体自卸车、矿用卡车、挖 掘机、装载机等）不同于常规的乘用车辆，它们本身的体积和重量非常大，采用的传动、转向等也存 在一定的特殊性，因而矿山移动设备存在惯性大、 系统时延大、载荷变化大等众多特点。此外，矿山 工地的工作环境恶劣、路面条件较为复杂，存在大 量连续爬坡、转弯以及路面不平等情况。因而无论 是整车特性，还是行驶环境，都比常规公路车辆更 加复杂，从而使设备的控制难度加大，需要控制算 法具备更好的自适应性和稳健性。
通过接收决策、规划、状态检测等系统信息， 单机无人控制可实现对期望轨迹的跟踪、行驶速度
的控制以及对挖掘、铲装和卸载货等的控制，其子 系统之间的关系如图 5 所示。
各部分的控制难点体现在以下几个方面。
• 对于行走过程中的横向控制，需要通过高精 度组合导航定位系统提供的设备实际信息、给定的 理想行驶轨迹和轨迹跟踪控制算法对转向系统进 行控制。由于转向方式多为全液压转向或履带式差 速转向，转向控制会存在一定的延迟或偏差；此外， 矿山道路不平也会引起设备定位产生偏差波动。因 此，普通的路径跟踪控制算法很难获得很高的跟踪 控制精度。
基于平行理论的平行转向控制[21]和基于虚拟 地形场的转向控制[22-23]具有克服上述缺点的先天 优势。平行转向控制不仅可以利用人类驾驶员的驾 驶经验，而且可有效降低对支持主动安全控制系统 的传感器和执行器的要求；基于虚拟地形场的转向 控制方法在路径跟踪控制中具备较好的自适应性和稳健性。
• 矿山设备行走中的纵向控制相比常规乘用 车辆更为特殊，如：电驱动矿卡同时具备了电制动 踏板和机械制动踏板，二者在不同工况下会交替发 挥作用，因而在速度控制过程中需要考虑更多因 素。对于电驱动行进设备，由于高速时的驱动/制动 力不足，在较大坡度下坡前必须将速度降至较低 值，否则会引起速度失控，进而发生危险。由于加 速和制动时间长，在遇见障碍物进行制动时必须保 持足够的反应距离和时间，才能保证安全。此外，还有弯道和管制道路等工况的速度控制问题。
• 挖掘和铲装过程主要是以规划、定位、感 知子系统的目标变量为基础，经过数据解析处理 后，通过挖掘机臂控制算法对接收到的信息进行计 算和处理，将挖掘机臂液压缸控制指令输出给相应 的执行机构，以达到对挖掘机臂的精准控制。通过 与运载车辆的协同实现自主装载控制。
• 卸载控制需结合车辆定位和环境感知子系 统，在进入卸载区域后，通过控制卸载举升缸油路， 实现对卸载过程的平稳、准确控制。同时根据对货 箱内剩余矿物的检测结果，确定卸载是否完成，若 尚未完成，则需要继续等待或控制车身移动加快卸 载剩余矿物；卸载完成后，车辆离开与货箱的回位 控制同步进行，以节省卸载时间。
5.4 多设备之间的协同作业
在实现矿物从地面到运 输设备的转移过程 中，需要挖掘机或装载机与运输车辆进行密切配合，才能准确迅速完成矿物的装载。以挖掘机与矿卡的协同作业为例，在矿物挖掘和装载区域， 由 于 挖 掘 机 在 挖 掘 过 程 中 的 位 置 会 不 断 发 生 变 化，矿卡从入场到停车等待装载的行驶路径需要不断进行规划。
装载区域各设备之间的协同作业流程如图 6 所示，主要分为进场前的路径规划、进场、倒车、装 载、离开等过程。具体过程说明如下。
首先，根据挖掘机当前所处的位置确定矿卡装 载时的停靠点 L，之后通过对挖掘机周边环境的感 知情况，并结合矿卡的几何特性，确定矿卡倒车起 始点 S。坐标点 I、O 分别为装载区域的入场点和离 去点。根据 I、S、L、O 点的坐标位置和航向信息， 结合周边障碍物和边界的感知结果，实现对装载区域矿卡行驶路径的规划。
为保证挖卡协同作业的高效进行，除了需要挖掘机与矿卡的通信外，还需要矿卡与矿卡之间以及管控中心与挖卡之间不断进行交互。无人矿卡在入场之前排队等待，从 I 点入场后在 S 点开始倒车至 L 点停车等待装运，在矿卡到达 L 点后，已完成挖掘。在此等待的挖掘机将铲斗内的矿物倒入矿卡货箱，之后继续装载直至装满。同时矿卡对自身载荷分配的情况进行自我检测，将检测结果通知挖掘机以进行动作调整，决定继续装载的位置。在装载的过程中，位于挖掘机臂前端的传感器实时检测货箱内矿物的装载状态，通过快速三维重建， 计算得出货箱内空间剩余情况，并指导挖掘机的矿物装载。完成矿物装载后，从 L 点开始离场，直到进入固定道路线。在装载区域需要管控中心控制在 IS、SL、LO 3 条线路上至少各有一辆矿卡运行，同时各矿卡之间还需不断交互，以防止发生碰撞。
5.5 平行矿山管控
平行矿山的管控中心主要体现为图 2 中的 E 点，同时在管控中心还会构建与实际系统一致的虚 拟矿山系统（F 点），用于模拟矿山的运行，指导和 提升实际系统。如图 7 所示，平行矿山管控中心主要实现三大功能：第一是矿卡和挖掘机的集群调度；第二是整个系统的故障监测；第三是最高权限 的远程遥控监管。
在矿卡和挖掘机的集群调度过程中，管控中心操作员主要负责分配挖掘机与多辆矿卡的协调合作权，实现一个挖掘机与多台矿卡的高效挖掘与运输模式。只需挖掘机驾驶员的半自主操作流程，就能实现多台矿卡的装载、运输和卸载。
整个系统的故障监测过程主要包括挖掘机的故障监管和报警、矿卡的异常情况报警和急停操作、无人矿卡或者无人货车的异常情况报警和急停操作，旨在通过管控中心监管员的快速、便捷操作来保证整个系统的安全和稳定。
在远程遥控监管过程中，有经验的驾驶员可以 在 管 控 中 心 对 监 管 的 无 人 矿 卡 或 无 人 货 车 提 出远程操控的请求，通过中心处理器的识别和处理，下发到特定的需要被控的车辆中，从而利用驾驶模拟器对远程车辆进行操控。这样做的目的主要是在车辆异常的情况下，通过人的操作来保证车辆的安全性，从而提升整个智能矿山系统的稳健性。
5.6 平行接管决策
远程接管平台用于在故障或者紧急情况下对 无人运输平台进行远程接管，平行无人矿山系统将 会涉及海量的无人设备监控，对应的远程接管驾驶 员也会有多个，对每个远程接管人员需要监管的设 备数量进行合理分配。若监管人员不足，无法保证设备的安全可靠运行，而监管人员过多则会造成人员、成本、资源的浪费。接管决策需要考虑的原则 说明如下。
• 需要根据设备的类型进行决断，不同类型的 设备需要分别监管，即每个人可以监管多个设备， 但只能是同一种设备。
• 设备操作越复杂，给每个人分配的监管设备 数量越少。
• 当出现需要接管的设备过多而无法全部接 管时，必须对不能立刻接管的设备进行制动停车，
以保证安全。
• 接管人员可保持适度的接管冗余。
平行无人矿山远程接管决策流程如图 8 所示， 具体过程说明如下。
首先需要根据设备是否发生故障以及是否出现控制偏差过大的紧急情况来判断是否需要接管；若需要远程接管，则根据位姿偏差、信号故障、车速、 加速度等参数计算设备的接管需求指数；通过比较所有需要接管车辆的需求指数大小，确定每个设备的接管优先等级；在所有监控车辆的接管需求及接管等级确定后，就可对需要被接管控制的车辆进行接管。对于最高接管等级的车辆（pi =1），直接将其 监控画面和控制权切换给接管员进行接管控制；若 此时还有其他车辆同时请求接管，则首先需要判断 是否还有其他接管员，若车辆的被接管等级 pi 值小于或等于接管员总数量 m，则将其监控画面和控制 权分配给其中的一个接管员控制；否则，说明没有 剩余的人员进行接管，此时必须对接管等级低的车辆进行制动/停车控制，确保车辆安全。当有接管员完成车辆接管而空闲后，即可对因人员不足而无法 接管的车辆继续进行接管，直到完成所有接管。
5.7 矿山网络通信
矿山地域广阔，环境恶劣，需要合理地设计和选择通信设备和通信方式，保证信息传输的安全性、可靠性。遮挡、空气含水率、设备部署等引起的信号衰减以及多普勒效应，会对无线通信的质量造成一定的干扰。对于传统 Wi-Fi，在网络拥塞时， 用户数据报协议（UDP）分组单向传输时间可能达到数百毫秒到数秒；对于无线网格（Mesh）网络， 随着跳数的增加，分组丢失、时延的概率增加，另外，网络拓扑会发生变化，并需要一定的时间收敛， 进而使得分组丢失、时延的概率进一步增加。
平行矿山的网络通信分为设备与中心之间的远程通信和设备之间的终端管理通信，方案如图 9 所示。在矿卡与挖掘机的协同通信中，Wi-Fi Mesh 通信设备分别安装在路侧、挖掘机、矿卡上，以实 现车到车（V2V）通信。客户端（CPE）通信设备 安装在挖掘机、矿卡上，与 4G 基站连接，实现车到服务器（V2Server）通信。
• Wi-Fi 链路中的策略：通过部署多通信系统、 链路预算冗余、加大频带和带宽的预算余量等，增强通信系统的可靠性，提高系统的抗干扰能力。结合各种通信协议的特点，根据实际场景和应用情况的不同，选取合适的网络传输协议。
•异常处理机制：在网络通信中不可避免地会 存在通信异常情况，如用户数据报协议（SUDP）发生分组丢失、车辆与控制中心的通信断开等。设计完善的异常处理机制，可以有效降低网络异常造成的各种影响[24]。异常处理方式有：将分组丢失情况告知管理员，并记录事件和日志；在通信断开时，及时向控制中心告警以通知管理员，或直接停车以确保安全。
6 结束语
在无人矿山建设方面，国外先进矿山设备制造商已实现了不同程度的矿山无人化运作。我国虽然在无人矿山的进程方面也取得了一定的进展，但与国外仍然存在较大差距，仍存在着诸多问题亟待解决。更重要的是，我国无人化矿山技术缺乏足够的 理论支撑，在知识产权等方面受限于国外。对此， 本文基于 ACP 方法提出了端对端平行无人矿山系统，给出了平行无人矿山的基本框架，并对其优势 进行了阐述；介绍了平行无人矿山的主要构成；详细论述了平行无人矿山的 7 个关键技术，并对各个关键技术的实现方法进行了说明，为无人矿山建设提供了一个新的思路。
在智能矿山的解决方案中，矿山机械无人化的实现将极大地提高设备的使用效率，减少人为误操作以及倒班时间，降低劳动成本；网络通信将增强车与车之间的协作，增加能源产量，提高生产率； 独创的中央平行管控系统能随时监测轮胎、电机等部件的运行，延长配件寿命，减少燃油费用，降低保养成本。矿区无人机械的集群化运营，将大幅提高矿山的运营效率。
参考文献
[1] 左敏. 平行系统理论在现代矿山管理中的应用[J]. 金属矿山, 2011, 40(9): 131-133.
ZUO M. The application of parallel system theory in modern mine management [J]. Metal Mine, 2011, 40(9): 131-133.
[2] 高磊. 基于平行系统理论的矿山安全作业系统的建立[J]. 金属矿山, 2013, 42(1): 113-115.
GAO L. Security operating system of mine safety based on the parallel systems theory[J]. Metal Mine, 2013, 42(1): 113-115.
[3] 苏子孟. 工程机械无人驾驶时代或将来临[J]. 工程机械, 2015.
SU Z M. Construction machinery unmanned era is coming[J]. Con- struction Machinery, 2015.
[4] How Komatsu’s autonomous haul trucks work and what it takes to imple- ment the technology at a working mine[N]. Equipment World News.
[5] EDNIE H. Automation for the underground [J]. CIM Bulletin, 2005, 1085(98): 12-13
[6] 陈晓晶, 何敏. 智慧矿山建设架构体系及其关键技术[J]. 煤炭科学技术, 2018, 46(2): 208-212, 236.
CHEN X J, HE M. Framework system and key technology of intelli- gent mine construction[J]. Coal Science and Technology, 2018, 46(2): 208-212, 236.
[7] 王飞跃. 关于复杂系统研究的计算理论与方法[J]. 中国基础科学, 2004, 6(5): 3-10.
WANG F Y. Computational theory and method on complex system[J]. China Basic Science, 2004, 6(5): 3-10.
[8] WANG F Y, ZHENG N N, CAO D P, et al. Parallel driving in CPSS: a unified approach for transport automation and vehicle intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2017, 4(4): 577-587.
[9] WANG F Y. The emergence of intelligent enterprises: from CPS to CPSS [J]. IEEE Intelligent Systems, 2010, 25(4): 85-88.
[10] 王飞跃. 平行系统方法与复杂系统的管理和控制[J]. 控制与决策, 2004, 19(5): 485-489.
WANG F Y. Parallel system methods for management and control of complex systems[J]. Control and Decision, 2004, 19(5): 485-489.
[11] 王飞跃. 软件定义的系统与知识自动化: 从牛顿到莫顿的平行升 华[J]. 自动化学报, 2015, 41(1): 1-8.
WANG F Y. Software-defined systems and knowledge automation: a parallel paradigm shift from Newton to Merton[J]. Acta Automatica Sinica, 2015, 41(1): 1-8.
[12] 王飞跃. 平行控制: 数据驱动的计算控制方法[J]. 自动化学报, 2013, 39(4): 293-302.
WANG F Y. Parallel control: a method for data-driven and computa- tional control[J]. Acta Automatica Sinica, 2013, 39(4): 293-302.
[13] 王飞跃. 人工社会、计算实验、平行系统——关于复杂社会经济系统计算研究的讨论[J]. 复杂系统与复杂性科学, 2004, 1(4): 25-35. WANG F Y. Artificial societies, computational experiments, and par- allel systems: a discussion on computational theory of complex so- cial-economic systems[J]. Complex Systems and Complexity Science, 2004, 1(4): 25-35.
[14] 白天翔, 王帅, 沈震, 等. 平行机器人与平行无人系统: 框架、结构、过程、平台及其应用[J]. 自动化学报, 2017, 43(2): 161-175. BAI T X, WANG S, SHEN Z, et al. Parallel robotics and parallel unmanned systems：framework, structure, process, platform and ap- plications[J]. Acta Automatica Sinica, 2017, 43(2): 161-175.
[15] 郑南宁. 人工智能新时代[J]. 智能科学与技术学报, 2019, 1(1): 1-3.
ZHENG N N. The new era of artificial intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(1): 1-3.
[16] 王飞跃. 平行矿山: 从软件定义的矿山到智能矿山[J]. 复杂性与智能化, 2018.
WANG F Y. Parallel mines: from software-defined mines to smart mines[J]. Complexity and Intelligence, 2018.
[17] 刘腾, 于会龙, 田滨, 等. 智能车的智能指挥与控制: 基本方法与系统结构[J]. 指挥与控制学报, 2018, 4(1): 22-31.
LIU T, YU H L, TIAN B, et al. Intelligent command and control sys- tems for intelligent vehicles: primary methods and systemic construc- tion[J]. Journal of Command and Control, 2018, 4(1): 22-31.
[18] 刘腾, 王晓, 邢阳, 等. 基于数字四胞胎的平行驾驶系统及应用[J].智能科学与技术学报, 2019, 1(1): 40-51.
LIU T, WANG X, XING Y, et al. Research on digital quadruplets in cyber-physical-social space-based parallel driving[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(1): 40-51.
[19] 张辉, 聂百胜, 许滕, 等. 煤矿虚拟现实技术应用与发展[J]. 煤矿安全, 2016, 47(10): 118-121.
ZHANG H, NIE B S, XU T, et al. Utilization and development of virtual reality in coal mines[J]. Safety in Coal Mines, 2016, 47(10): 118-121.
[20] 赵翾. 无人驾驶地下矿用汽车路径跟踪与速度决策研究[D]. 北京: 北京科技大学, 2016.
ZHAO X. Research on path tracking and speed decision of unmanned underground mining vehicles[D]. Beijing: University of Science & Technology Beijing, 2016.
[21] GUO H, SONG L, LIU J, et al. Hazard-evaluation-oriented moving horizon parallel steering control for driver-automation collaboration during auto- mated driving[J]. IEEE/CAA Journal of Automatica Sinica, 2018.
[22] GAO Y, SHEN Y, ZHANG W, et al. Robust path tracking control for autonomous heavy vehicles[J]. SAE Technical Paper, 2018, 1: 1082.
[23] 高玉. 铰接式工程车辆操纵稳定性控制与可视化模型开发[D]. 北京: 北京科技大学, 2018.
GAO Y. Handling and stability control for articulated steer vehicles and virtual reality model development[D]. Beijing: University of Science & Technology Beijing, 2018.
[24] 王晓, 要婷婷, 韩双双, 等. 平行车联网: 基于 ACP 的智能车辆 网联管理与控制[J]. 自动化学报, 2018, 44(8): 1391-1404.
WANG X, YAO T T, HAN S S, et al. Parallel Internet of vehicles: the ACP-based networked management and control for intelligent vehi- cles[J]. Acta Automatica Sinica, 2018, 44(8): 1391-1404.
Parallel End-to-End Autonomous Mining: An IoT-Oriented Approach
Abstract—This paper proposes a new solution for end-to-end autonomous mining operations: Internet of Things (IoT) based parallel mining, consisting of the concept definition, the solution given and the concrete realization. The proposed parallel mining is inspired by the ACP (artificial societies (A) for modeling, computational experiments (C) for analysis, and parallel execution (P) for control) approach. The basic framework of parallel mining is given and its advantages are expounded. Then the solution of parallel mining is proposed, which is mainly composed of four parts: the management and control center for autonomous mining, the autonomous transportation platform of truck, the semi-autonomous mining / shovel platform, and the remote take-over platform. Key technologies of IoT based parallel mining are discussed in detail, namely, network communication, virtual parallel mining construction, mining environment perception over-the-horizon for the moving area and obstacle detection, collaborative decision-making, planning and control for unmanned mining equipment, parallel taking-over and remote control. Finally, the performance of IoT based parallel mining, including fusion perception, collaborative decision-making, planning and control, are evaluated. The realization of parallel mining can fundamentally improve the safety of personnel and equipment, reduce the cost of mining operation and increase the production rate.
Index Terms—Internet of things, ACP approach, parallel mines, management & control, decision-making, autonomous truck
I. Introduction
Today, the world has entered a new era of intellectualized development. Cyber, physical and social systems can be integrated by the Internet of Things (IoT) which enables the perception, computation and execution in an intelligent design paradigm [1]. Under such a background, the mining industry is facing many challenges, such as relatively low price of mineral products, deeper exploitation of resources, poor working environment, etc. [2,3]. In addition, the serious aging of employees, the shortage of skilled workers, the rising cost of manpower and the stricter requirements of safety and environmental protection have brought great challenges to the development of mining industry[4, 5].
As the driving force of the automobile industry reform, auto-driving technology has not only become the focus of many traditional automobile enterprises all over the world, but also one of the focuses of major Internet giants. However, due to limitations of policies, security, technology, ethics and many other factors, the commercial success of unmanned vehicles in the open scenario on public roads is facing many bottlenecks. Noticeable obstacles includes complex traffic signs, unpredictable behavior of road participants and unstable hardwares [6]. While general purpose autonomous driving suffers from these difficulties, specialized applications for mining sites could be realized relatively easily and provide novel solutions to above mentioned challenges in the mining industry. On the one hand, by connecting the vehicles to the Internet of Things, they can transmit all kinds of information to the central processing unit to realize the intelligent traffic management and vehicle control [7]. On the other hand, the special vehicles under limited scenarios with low or medium speed conditions will be the first rigid requirement for auto-driving technology to commercialize. As a relatively closed scenario, mining system has become one of the most important areas concerned by many auto-driving related enterprises [8,9]. As the mining and transportation operation in the mines implied a degree of repeatability and have some commonly used conditions, the construction machineries, such as excavators, wheel loaders, mining dump trucks, etc., first started the process of unmanned refit [10,11].
Unmanned mining has huge market potential, which can bring the mining enterprises (users) with cost, safety, efficiency and environmental benefits. Correspondingly, it will bring huge profits to the mining equipment manufacturers. It has attracted competing investment from mining equipment manufacturers, self-driving technology companies and large mining enterprises. However, the proportion of mines using automated equipments is still very low at present. With the pursuing for zero casualties and entering the era of skilled workers shortage in developed countries, it is estimated that by 2020, 50% of the major mining companies in Australia, Europe and the Americas will use the unmanned products, and some mines have decided to adopt unmanned mining equipments wholly. [12]
To realize the end-to-end autonomous mining operations, Komatsu launched the Autonomous Haulage System (AHS) to operate and manage fleets of self-driving mining trucks with capacities between 200 and 400 tons [13]. Caterpillar released Cat MineStar system to optimize productivity, enhance security, and improve machine utilization [14]. Sandvik developed autonomous loaders and trucks which operate underground [15]. Volvo tested it’s self-driving tipper inside the earth at a depth of 1,320 meters and even with artificial lighting [16].
Overall, it has been possible to realize the autonomous driving on a single mining equipment no matter in an above-ground mine or an underground mine. However, more researches are needed on the control robustness of unmanned equipment, on the cooperative operation among equipment and on the cluster operation to achieve the overall improvement of safety, cost and efficiency of unmanned mines. As a typical complex system, mine system includes production subsystem, dispatching subsystem, water and power supply subsystem, transportation subsystem, management and control subsystem, safety subsystem etc. Of which production, transportation, dispatching and safety are the key subsystems in realizing the unmanned operation of mining system [17]. Faced with the control and management of such a complex and huge system, parallel theory [18,19] and ACP approach [20-22] provide a good solution for mining intellectualization and unmanned operation. The parallel mines inspired by parallel theory and ACP approach is a highly networked intelligent system, which is in the scope of IoT. The IoT based parallel mining also refers to humans, excavators, off-road dump trucks, high-speed heavy trucks and other devices connected in the mining environment.
By using IoT technology, the real-time data of key systems in mining are collected and transmitted to the management center, where a dynamic artificial system corresponding to the real system are constructed. This artificial system is proposed to simulate the real complex system in real time, dynamically and on-line. By studying the evolution and prediction of artificial systems, the prediction and management of real mining systems are realized.
The main contributions of this work include: 1)The proposing of the framework and solution for IoT based parallel mining, which may improve the safety of personnel and equipment, reduce the cost of mining operation and increase production rate by improved efficiency. 2)The key technologies motivated by IoT thought [23-25] in the implementation of autonomous mining are introduced: from the network communication, model construction, integrated perception, to the collaborative decision-making, take-over and control.
The rest of the paper is organized as follows. Section II describes the framework of the IoT based parallel mining. Section III gives the solution for the realization of parallel mining. The key technologies of parallel mining are introduced in Section IV in detail, which mainly include: the network communication, virtual parallel mining construction, mining environment perception, collaborative decision-making, unmanned mining equipment control, as well as parallel taking-over and remote control. Section V evaluates the performance of the key technologies based on Internet of Things in parallel autonomous mining. Finally, our conclusion is provided in Section Ⅵ.
II. The Framework of Iot Based Parallel Mines
The concept of parallel system was first put forward by F. Wang in 2004 [26], that is, to study the control, management, optimization and guidance of complex systems by constructing the virtual-real interactive artificial system and real system model. In parallel system, the ACP approach is the hard-core technical section, in which A refers to artificial society, C refers to computational experiments and P refers to parallel execution [27,28].
As a relatively closed and structured environment without interference from foreign vehicles and pedestrians except for operating equipment and professionals, mining system becomes one of the main scenarios in which driverless technology can be realized quickly. The driver's high labor intensity and poor working conditions also promoted this progress. Compared with the unmanned operation of single equipment, the cluster cooperative unmanned operation among all the mining equipments is more needed, which is exactly the important solution provided by parallel driving [27]. Parallel driving could obtain the data and scenarios in perceptual limits and other situations via the describing system. Knowledges and experiences can be shared to all vehicles in the system through analysis of a large number of algorithms and large-scale computation and evaluation of driving decisions. Relying on a single vehicle’s own ability of perception, decision-making and control, as well as the powerful GPU behind it for data processing, the vehicle itself possess the ability to adjust its own behavior according to the environmental changes. The human remote control is only activated when the early warning had occurred. The IoT based parallel mining is an intelligent solution that applies parallel driving technology to the mining scenarios.
The framework of IoT based parallel mining is shown in Fig.1. It is a true embodiment of parallel system in the field of mining. The parallel mines has five important locations, namely the loading site A, the dumping site B, the transfer belt unloading site C, the freight station D, the parallel management & control center E. Among them, A, B, C and D can be regarded as real systems, while E and F can be regarded as virtual systems.
Within the parallel mining system, we mainly study and solve four problems: the first is the semi-autonomous excavation at point A, which improves the efficiency of excavation through the cooperation between excavator and mining truck. The second is the autonomous transportation of off-road dump truck from point A to point B. By installing sensors and developing algorithms of intelligent sensing, decision-making and control, the dump truck could realize it’s autonomous driving. The third is the autonomous transportation of high-speed heavy truck from point C to point D. Unlike off-road dump truck, the high-speed heavy truck is operated on the open road and running in a relatively high speed. It requires higher level automated driving capacity which could avoid possible obstacles. The fourth is the management and control of parallel unmanned mining, which is responsible for the communication, supervision, coordinated control and dispatch for the automatic operation of all equipment. The end-to-end unmanned mining operation can be realized by the cooperation of these four sections, which aims at improving the safety and efficiency of the whole system and reducing the cost of mining operation.
Fig.1 The framework of IoT based parallel mines
In the parallel mining, parallel subsystems corresponding to their real counterparts will be established to realize the physical system in real space mapping into virtual space. It could essentially solve the problems of low degree of network connection, difficult scheduling and confused management of each subsystem. Driven by data, a series of virtual mining organizations and institutions that interact with the actual mining system are constructed to form the parallel unmanned mining system. Various complex problems including mining scheduling, coordination, management, control and communication are constantly analyzed and evaluated by means of computational experiments. The aim is to change the production mode of the mining system and improve the synergy level and intelligence degree between the mining subsystems. The parallel mining divides a complex huge mining system which is difficult to manage, control and forecast into several parallel subsystems according to its functions, which can be flexibly optimized. At the same time, the forecasting and management strategy explored in the virtual world through computational experiments and parallel execution can be applied on the real mining system, enabling the real mining system’s ability of coping with unknown work scenarios and tasks.
III. Solution for Iot Based Parallel Mines
The IoT based parallel mining system consists of four parts: the parallel management and control center (hereafter called M&C center), the autonomous transportation platform, the semi-autonomous mining/shovelling platform, and the remote take-over platform. As the nerve center of the mining system, the parallel M&C center is responsible for comprehensive real-time monitoring and comprehensive dispatching of unmanned mining equipment. The real-time interaction and two-way optimization through the established virtual mining system and their real counterparts ensure the safe and efficient operation of actual mining. The semi-autonomous mining/ shovelling platform and the autonomous transportation platform are equipped with a variety of sensing devices to realize semi-autonomous excavation, loading, unmanned transportation and unloading of minerals. The remote take-over platform is responsible for monitoring the operation of the mining equipment. Under the initiative request of the takeover from the semi-autonomous mining/shovelling platform or autonomous transportation platform, or in a special emergency, the remote take-over platform realizes the intervention and takeover for multiple devices according to the decision made by the parallel M&C center.
Fig.2 Solution for IoT based parallel mines
A. The parallel M&C center
The parallel M&C center consists of a description system defined by the virtual parallel mining system, a prediction system defined by big data analysis and the deep learning center, and a guidance system defined by the dispatch center. Various data generated during mining production and operation are transmitted to the information processing center, where each subsystems parallel to the actual mining operation are constructed through data fusion, data mining, and visualization processing. In this way, on the one hand, the virtual data are built using actual data to complete parallel, online and real-time simulation of all aspects of mining production operations; on the other hand, the prediction, evaluation and optimization of real systems are completed through the evolution of parallel systems and collaborative interaction with real systems, thereby achieving the management and control of the unmanned mining system. The hardware in parallel M&C center includes simulation equipment, industrial computers, servers, video monitoring devices, image splicers, remote network equipment, etc. Each device corresponds to the function of the control center. In particular, if a dangerous situation occurs at the mining site which may affects the normal operation of other systems or the safety of workers, the control center will make intelligent decisions and monitor the mining site in real time.
B. The autonomous transportation platform
The autonomous transportation platform mainly includes off-road dump truck and high-speed heavy truck, which are used for short and long distance transportation, respectively. The autonomous truck is equipped with lidar, radar, inertial navigation system, camera, V2X equipment, industrial computer, wireless network tester, on-board monitor, etc. The transportation route of autonomous truck is assigned by the parallel M&C center. The truck runs automatically at a suitable speed in the cycle of loading, transportation and unloading according to the target route, its own position and the surrounding environment.
The autonomous transportation platform includes functions such as autonomous driving, mode switching, status detection and display, information communication and management. The mining environment information perceived through lidar, radar, camera, as well as the positioning information provided by the inertial navigation system, are provided to the decision-making subsystem and control subsystem for the decision, planning and control. The communication subsystem is responsible for the wireless communication between the unmanned equipment, the M&C center and the remote take-over platform, thus providing support for the remote monitoring and control, and the cooperation between excavators and mining trucks. The data management subsystem performs data backup for the operation of autonomous truck, which can be used for data playback and further research. Considering the special requirements of mining operation safety, the autonomous transportation platform also has a state detection subsystem and a mode switching subsystem. The state detection subsystem displays the health status of each section. Integrating the data and commands from the state detection subsystem, the M&C center, and the remote take-over platform, the mode switching subsystem make decisions on the switching between manual mode and the automatic mode.
C. The semi-autonomous mining/shovelling platform
The semi-autonomous mining/shovelling platform is equipped with sensors such as displacement sensor, pressure sensor, lidar, inertial navigation system, camera, radars, etc. The excavators or loaders cooperate with the autonomous truck by semi-autonomous mining or autonomous shoveling at mining sites.
In the process of the cooperation between excavators and mining trucks, the driving path for the mining truck in the working area is firstly planned according to the position of the excavator, and the excavator guides the mining truck to the correct position for loading. Then the excavator fulfill the automatic excavation of target minerals [29,30] and automatic unloading into the container of the mining truck according to the sensor information. During the entire excavation process, the human operator in the control center could monitor the excavation process and manually intervene when necessary, which not only ensures the smooth progress of the automatic excavation, but also greatly improves the working environment of the driver.
For the cooperation between the loader and the mining truck, after the mining truck is guided to the target position, since the loader moves more conveniently, the self-loader is responsible for the mineral loading, driving route planning and tracking, and finally depositing the minerals into the truck container.
D. The remote take-over platform
To fulfill the high safety requirements for unmanned mining operation, the remote take-over platform is responsible for the safe manual control for the autonomous mining truck and the semi-autonomous excavators/loaders at emergency situations [31]. The remote take-over platform implements different processing strategies according to the fault levels, and designs a multi-level policy of security confirmation for mode switching and takeover. The remote take-over platform can realize the control for multiple autonomous mining trucks or excavators/loaders. It could also realize the cooperation between the excavator and the mining truck in the remote take-over state through the monitoring and dispatching of the M&C center. The remote take-over platform improves the operation quality and productivity while increasing the safety of human drivers and the equipment.
IV. Key Technologies of Iot Based Parallel Mines
This section introduces the key technologies we studied in the implementation of IoT based parallel mining, which mainly includes: the network communication, virtual parallel mining construction, environment perception, the collaborative decision-making, unmanned equipment control, as well as parallel taking-over and remote control. The details are as follows.
A. Mining network communication
Due to the influence of signal attenuation and Doppler effect caused by occlusion, air moisture content, equipment deployment, etc., the quality of wireless communication will be disturbed to a certain extent. It is necessary to design and choose a reasonable communication mode and equipment to ensure the security and reliability of information transmission. For traditional WiFi, the one-way transmission time of UDP packets may reach hundreds of milliseconds to seconds when the network is congested; For Mesh wireless network, the probability of packet loss and delay increases with the increase of hops, and the network topology will change and needs a certain time to converge, which further increases the probability of packet loss and delay.
To ensure the reliability and stability of network communication, we use a combined network scheme of WiFi-Mesh in parallel mining network communication. The principle of the network communication in parallel mining is shown in Fig.3, which is divided into the remote communication between equipment and center and the terminal management communication between different equipment. In the cooperative communication between mining truck and excavator, the Wifi-Mesh communication equipment is installed on roadside, enabling excavator and mining truck to realize the V2V communication. The CPE communication equipment is installed on excavators and mining trucks, and connected with the 4G base station to realize the V2Server communication.
In order to enhance the reliability and the anti-jamming ability of communication system, certain strategies, such as deploying multi-communication system, increasing link budget redundancy, increasing the budget margin of frequency band and bandwidth, are adopted in the WiFi link. The network transmission protocol can be selected according to the characteristics of various communication protocols, scenarios and applications.
Fig.3 The principle of mine network communication
In the network communication, there will inevitably generate communication abnormalities, such as SUDP packet loss, the communication disconnection between vehicle and control center, etc. A well-designed exception handling mechanism can effectively reduce the impact of network anomalies. Abnormal handling methods include: displaying the loss of packets to the administrator, recording events and logs; alarming the control center in time to inform the administrator when communication disconnection occurs, or parking directly to ensure safety.
B. Virtual parallel mining construction
As the foundation of parallel M&C center of unmanned mining, the construction of virtual parallel mining is essential. In practice, the mining system is complex and changeable, and it is difficult to build a reliable mathematical model. Therefore, the mine production process is difficult to be controlled accurately, and of great safety hazard. To overcome these difficulties, information technology is used to collect data of key dynamic attributes for each subsystem in mining system. Besides, digital modelling, virtual technology and visualization technology are used to establish artificial system corresponding to the system in reality, and the collected data are imported into the artificial system in real time, making the artificial system parallel to the real system, which can be called as parallel system.[32] Furthermore, in the parallel systems, various intelligent algorithms and optimization algorithms are used to predict and optimize the operation of the real system, and send the results to the real system, so as to realize the optimal control of the real system.
Taking the autonomous truck as an example, the structure of virtual parallel mining is shown in Fig. 4, and its visual display scene is shown in Fig. 5. The autonomous truck simulation platform in virtual parallel mining covers the module of vehicle dynamic model, virtual reality model, virtual sensor and environment perception, deviation calculation, planning and decision-making, control, etc. The output data include camera videos from each view point, radar data, 3D coordinates of vehicles, heading angle, speed, steering wheel angle, throttle/brake pedal’s position, tire forces and so on.
The input of the platform is the relevant data generated by the actual truck, including the current position, vehicle speed, the steering command from the controller, and the pedal instruction. After the collected data have been imported into the artificial system in real time, the autonomous truck in virtual parallel mining is synchronized with the real truck. With this platform, we can accurately simulate the truck performance and predict possible problems. It can also quickly verify the effectiveness of the autonomous driving algorithms in planning, decision-making and control, especially in various extreme conditions. Finally, the prediction, control and management strategies will be applied to the real mining system.
Fig.4 The structure of virtual parallel mine
Fig.5 Visual display of parallel mine
C. Perception over-the-horizon in mining Environment
The environment perception system of each mining truck uses a collection of sensors, such as lidar, millimeter-wave radar, camera, ultrasonic wave sensor, infrared sensor, etc., to perceive the environmental information. During the operation of mining machines, multi-sensor fusion technology is used to monitor the driving area in order to detect and track the obstacles. At the same time, it can identify specific targets to avoid collision and provide warning information while reversing.
Multi-sensor data fusion is also used to eliminate the data redundancy between different sensors, and enhance the robustness of the system through data complementarity, so as to improve the reliability of the environmental sensing system. The sensors and their coverage areas of the mining truck are shown in Fig.6. The overlapped coverage areas of multiple sensors are the working areas of data sensing and fusion.
Fig.6 Sensors & it’s coverage of the mining truck
The resolution of sensors used in our truck is usually not very high, which reduces the capital cost, at the expense of moderate perception ability lose. In addition, the road width in the mining area could easily reach 30-40 meters, and the effective perception range of sensors in dust/sand environment is greatly reduced. Therefore, it is difficult to make safe and efficient decisions based on the perceptual results of a single vehicle in complex traffic scenarios. Thanks to the cluster management and control system and V2X communication technology, all perceptual results, such as vehicle localization, status information and surrounding environment perception information, can be uploaded to the cluster center. The center can further send the perception results of other vehicles, within a certain range around it, to a vehicle, so as to realize the sharing of perception results among multiple vehicles and achieve the ability to perceive over-the-horizon. The principle of fusion perception over-the-horizon is shown in the Fig.7.
Fig.7 Perception over-the-horizon by sharing
D. Collaborative decision-making
Based on the perception results of over-the-horizon for mining environment acquired via V2X communication, the scheduling and management of the fleet in complex mining scenes can be realized by designing the decision-making algorithm for autonomous mining trucks. In the parallel autonomous mining transportation, the collaborative decision-making between the management & control center and autonomous truck is designed, which could greatly improves the efficiency of transportation for each autonomous truck. In order to ensure the reliability of the unmanned system, each autonomous truck still maintains its own decision-making within the range of its perception ability. In addition to emergency scenarios, the decision-making instructions issued by the center have the highest priority. If the center does not issue any instructions, the decision-making by the autonomous truck is used.
In areas beyond the perception range of a single vehicle, the decisions can be made by the M&C center, or by each autonomous truck according to the information from other vehicles that is issued by the center. The comparison of two decision-making modes is shown in Fig. 8. When a decision is made by the center, all vehicle parameters are calculated only once at the central terminal, and the processing speed of decision-making is faster because of the higher central computer specs. When a decision is made by each autonomous truck, the center needs to send the perception/location information of surrounding vehicles to each vehicle, which not only increases the amount of data transmission, but also increases the network communication burden. And the decision-making process is slower because of the limited computing power of the calculation unit mounted at each vehicle, and the multiple calculations carried out on each vehicle. For example, assuming that there are n autonomous truck in the whole mining area, all the positioning and sensing information of the truck is uploaded to the central terminal. The network occupancy rate is very low when the decision is made by the center, which does not need to send a large number of vehicle perception information to individual vehicles. Only one calculation was made at the central terminal, and then the decision-making instructions is sent to individual vehicles. However, if the decision is made by individual autonomous trucks, each vehicle needs to do a decision-making calculation (totally n times of calculation) in addition to sending a large number of perception data to surrounding vehicles. Moreover, if the decision is only made by autonomous trucks, the traffic problem among the autonomous trucks can not be effectively coordinated.
Therefore, the integrated dispatch of all vehicles through the fleet management system at the central terminal has the advantages of fast decision-making speed and low network burden, which can effectively improve the transportation efficiency of autonomous trucks and ensure the traffic safety.
After all the vehicle perception information is uploaded to the central terminal for decision-making, the fleet management of autonomous truck is transformed into the decision-making and planning for traffic flow. Here, the decision-making at the intersection is taken as an example to illustrate. The road in the mining area is specially designed. Traffic lights are usually not used at intersections, so the “first come first go” traffic rules are adopted. Under this traffic rule, vehicles need to slow down or stop at the intersection before passing, which affects the transportation efficiency of mining area and increases the fuel consumption of transportation. In parallel autonomous mining transportation, the M&C center predicts, plans, and manages all vehicles passing through the intersection in advance, automatically manages the order and speed of vehicles, avoids collision or parking at the intersection, and enables all vehicles to pass through the intersection in efficient and smooth motion.
Fig.8 Comparison of decision-making made by center and by vehicle
The principle of collaborative decision-making at intersection is shown in Fig. 9. Two circular areas are designed in the intersection area: the rescheduling area between outer cycle and inner cycle for passing order and speed planning, and the crossing area within inner cycle with determined rules. When a new vehicle enters the outer circle in a certain speed, the velocity and passing order of all vehicles within the annulus will be rescheduled. Once a vehicle enters the inner circle, its planned passing order and speed will not be changed for safety reasons. It should be noted that the decision-making algorithms should arrange the velocities of each autonomous truck to ensure that they will not collide in intersection area. [33,34]
E. Unmanned mining equipment control
Mining equipment, such as off-road dump trucks, high-speed heavy trucks, excavators, loaders etc., are much different from conventional passenger vehicles. Their size and weight are very large, and the used transmission and steering systems also have certain particularities. Therefore, the mining equipment have many unique characteristics, such as large inertia, large system delay and large variation of load. In addition, the working environment of the mining site is harsh and the pavement conditions are complex. There are a lot of continuous climbing, turning and uneven pavement. Therefore, both the facility performance and working environment are more complex than conventional passenger vehicle’s scenario. It makes the control of these equipment more difficult, which requires the control algorithm to have better adaptability and robustness.
Fig.9 Collaborative decision-making at intersection
For the lateral control in the course of driving, the steering system needs to be controlled by the path following control algorithm according to the information provided by the integrated navigation and positioning system and the planned ideal trajectory. The path-following control based on the virtual terrain field (VTF) method is used for the heavy mining truck control [35,36]. For the VTF control, a virtual U-shaped terrain field is assumed to exist along the reference path. The altitude of terrain field will goes higher as the lateral error becomes larger. If the vehicle deviates from the reference line, additional lateral restoring forces caused by the virtual banked road will be applied on the vehicle. The vehicle will be pulled back to the lowest position (reference centerline) under the influence of the additional lateral forces.
Fig.10 The processing flow of VTF control
Fig.10 is the processing flow of VTF control. First, the tracking errors is calculated according to the digital map and parameters acquired by sensors. Then the function of VTF altitude and the lateral tire forces are established and derived, respectively. According to the 2-DoF vehicle dynamics model, the target steering angle are derived, which is applied to the vehicle for path-following control.
F. Parallel taking-over and remote control
The large-scale unmanned mining is a centralized operation business scenario, in which thousands of engineering machines (including mining truck, semi-autonomous excavator, patrol cars, etc.), surveillance cameras and various sensors, etc, are used. It should be pointed out that the unmanned mining scenario is not completely unmanned operation. Its main meaning is the unattended operation and intelligent decision-making for mining. Therefore, in the current planning, a certain number of professionals are still needed to manage and assist decision-making through appropriate takeover or remote control, so as to maintain the normal operation of various equipment.
In the remote supervision, the experienced drivers can send takeover request for the supervised mining trucks in the control center. Through the identification and processing of the central processor, the command can be sent to the specific trucks that need to be controlled, so as to realize the remote control by driving simulator. The main purpose of the remote control is to ensure the safety of the truck through human operation in the case of abnormal condition, so as to enhance the robustness of the whole intelligent mining system.
For parallel unmanned mining system, a large number of unmanned equipment and a certain amount of takeover drivers are need. The specific number of equipment needed to be supervised by each driver needs to be allocated reasonably. Insufficient supervisors can not guarantee the safe and reliable operation of equipment, and excessive supervisors will cause waste of personnel, costs and resources.
The decision-making process of remote takeover for parallel unmanned mining is shown in Fig.11. The detailed process is as follows:
Firstly, it is needed to judge whether the takeover is necessary according to the emergency situations such as the break down of the equipment or large path tracking error. If the remote takeover is needed, the take-over index of the equipment is calculated according to the parameters such as position deviation, signal fault, speed and acceleration. By comparing the demand index of all the equipment that need to be takeover, the priority grade of takeover of each equipment is determined. After the takeover index and the priority grade of all monitored equipment are determined, those equipment that need to be takeover can be controlled. For the equipment with the highest priority grade of takeover (pi=1), the monitor screen and control are directly allocated to the takeover driver for remote control; If other equipment request to takeover at the same time, it is necessary to determine whether the driver is sufficient. If the takeover grade of the equipment is less than or equal to the total number of takeover drivers, the monitor screen and control are allocated to one of the drivers. Otherwise, there is no remaining driver to takeover. At this time, braking/parking control must be carried out on the equipment with low level of takeover to ensure safety. When a driver completes the takeover for an equipment and is idle, the stopped equipment that cannot be takeover due to insufficient driver can continue to takeover until all the equipment are completed.
Fig.11 The decision-making process of remote takeover
In the way of remote takeover, due to the difference of the specific operation, the control mode of different equipment is also different. For driving-oriented equipment (such as mining truck), the control mode is relatively simple, and remote control can be carried out by driving simulator [31]. For excavator, loader or other equipment, excavation, shovelling and loading operations are also needed besides driving. This paper takes the remote gesture control of excavator as an example to illustrate:
The remote operator of excavator wears the wearable equipment on his upper body as shown in Fig.12, which can measure the three-axis angle of the arm, and then calculate the spatial position of the hand and some arm postures. The relevant control program is designed to realize the movement control of the excavator arm bucket in tracking the spatial position of the human hand and the action control of each mechanism of the excavator arm. The movement of the excavator includes the swing and lifting of the main boom, the lifting of the stick boom and the swing of the bucket. The left hand is responsible for switching gears, and the right hand is responsible for specific displacement control. The specific implementation scheme is shown in Fig.12.
The three kinds shift switching control of left-handed attitude are tracking control (A), stopping control (B) and separate mechanism control (C). When tracking control, the left hand sags naturally, which is used to track the spatial position of the human hand in real time by the excavator bucket; the left arm is placed in front of the chest to indicate that the excavator arm stops moving; and the mechanism is controlled separately to indicate that the four main movements of the excavator are controlled separately.
Fig.12 The remote gesture control of excavator
Thanks to the virtual parallel mining and equipment models which correspond to the real system, the video around the equipment is no longer need to be retransmitted during remote operation, but only the main parameters such as the position and attitude of the equipment are sent to the remote takeover terminal. Based on the limited data, the virtual equipment model is directly driven and displayed to the operator, which is used for the remote operation. As there is no need for video data transmission, the general 4G network can meet the network bandwidth requirements and achieve better control effect. Otherwise, it will cause a greater pressure on network communication, and 5G must be used to meet the demand.
V. Performance Evaluation of Iot Based Parallel Mines
This section evaluates the performance of the key technologies based on Internet of Things in parallel autonomous mining, including the fusion perception, collaborative decision-making, planning and control.
A. Perception
The detection module can collect information of road surface, driving area and obstacles, and provide reliable guarantee for safe and fast autonomous navigation. For road surface and driving area detection, lidar, camera and other sensors are used to extract roads and driving areas, so that mining machinery can reach the target point safely within the driving area when GPS positioning fails. As for target detection and tracking, by processing the sensor data from lidars and cameras, the obstacle information is detected and the dynamic obstacles are tracked. This information is transmitted to the planning module to realize the autonomous obstacle avoidance.
As the road for mining truck running is the unstructured type, it makes the road detection much difficult. First, the point cloud data of lidar is processed to eliminate the noise interference and to determine the grid map [37,38]. Then DBSCAN clustering is conducted to extract the ground point [39]. As the features of scanning points in vertical direction are distinct after dilation operation [40], if the difference between the maximum and minimum value of each grid is less than a set threshold, the grid is considered to be the ground, otherwise it is considered an obstacle. An example result of road pit detection is shown in Fig. 13 (a).
Ground points are fitted with Hough transform so as to extract the straight lines from laser scans on the ground and the end points of lines are taken as the contour points. Quadratic curve fitting with least square algorithm is used to find the road boundaries. The real detection results for road boundary and surface of the unstructured road are shown in Fig. 13 (b), (c).
The clustered obstacles, including pedestrians and vehicles, are fitted with boundaries. As the two-dimensional shape of the vehicle is rectangular, the L-shape fitting algorithm [41] is used. The minimum area rectangle (MAR) method [42] is used for pedestrian detection. The detection results are shown in Fig. 13 (d), (e).
(a) road pit
(b) road boundary (c) road surface
(d) pedestrians (e) vehicles
Fig.13 Detection for road and obstacle
B. Decision-making & planning
(1) Decision-making at intersection
To verify the effectiveness of collaborative decision-making for the fleet management of autonomous mining trucks, the whole time to pass through an intersection with different arriving rate λ (traffic load) are simulated. The results comparing with "ad hoc negotiation based" strategy are given in Fig.14. The testing arriving rate of autonomous truck is varying from 0.05veh/(lane•s) to 0.9veh/(lane•s). When the traffic arriving rate is very low (e.g. λ=0.05veh/(lane•s), the performance of the collaborative decision-making strategy is similar to that of "ad hoc negotiation based" strategy. This in mainly because the vehicles pass through the intersection do not need to stop when the traffic rate is low, so the passing time are basically the same. But when the traffic arriving rate goes higher, the time to pass through the intersection by the "ad hoc negotiation based" approach increases much faster than that of the collaborative decision-making strategy, which means the collaborative decision-making strategy has a notably advantage in leading to a quick and efficient passing performance. [43]
Fig. 14. The time to pass through an intersection with different traffic load [43]
(2) Local speed planning
Once the start and end points for the operation are determined, the fleet M&C center distributes the global path and speed planning results to each autonomous mining truck. However, it is inevitable to suffer discontinuous region of the planned global speed. When switching between different speed regions, the large speed deviation could result in sudden acceleration or braking of vehicles. For urgent acceleration, the target value can be gradually achieved through continuous acceleration without dangerous. However, the urgent braking must be accurately executed immediately for extra braking distance could bring about collision and other dangers. Therefore, the local speed planning for the braking buffer area must be determined before the discontinuous region of speed, and the braking begins when the vehicle enters the area.
To verify the effectiveness of the local speed planning algorithm, the local speed planning and control for mining truck on circular road was simulated on virtual simulation platform. The preliminary global speed planning result with discontinuity is shown by the grey curve in Fig.15. It can be seen that there is a sudden change in the planned speed value at the junction of straight line and turning, especially during the braking at 10s and 41.3s. If not replanned to decelerate in advance, the mining truck will rush into the turning area at a higher speed, which will affect the comfort and even safety of the truck. Under the established local speed planning algorithm, the mining truck can be accurately braked in advance before 10s and 41.3s, respectively. When the truck arrives at the turning area, it can exactly reach the preliminary planned velocity (about 20km/h).
The local speed planning takes into account the braking ability of the mining truck by controlling the deceleration according to the initial planned speed within the preview distance and the current speed. The braking related operations can be carried out timely and accurately. As the actual power of the truck is considered, the driving speed changes smoothly and the maximum deceleration can also be limited, which improves the stability, comfort and safety of the mining truck.
Fig.15. The local speed planning to solve discontinuities
C. Control
In view of the particularity of the mining equipment and their working environment, if the control algorithm is directly verified on the real facilities, it may cause unnecessary risks, increase the waste of humans and costs. With the virtual simulation platform, the control algorithm for unmanned equipment can be quickly, safely and fully validated. Simulations of steering control by VTF approach at annular road, double-lane change road and snaking road were conducted to compare with PID control and preview feedback control (PFB) [44].
Fig. 16. Results comparison by different type controllers
To evaluate the driving performance of different controllers, the average lateral error and the average driver steering load under different scenarios are compared in Fig. 16. The results show that the average lateral errors controlled by VTF approach has significant advantages with the minimum value under the three scenarios compared with the other two scenarios. Compared with PID control and PFB control, the VTF approach could reduce the lateral error by 61%~88% and by 36%~74%, respectively. At the same time, the steering load of all the control algorithms are basically the same under all scenarios.
In addition, we compared the computer resource consumption during the process of the steering control to evaluate the real-time performance of different algorithms. Besides PID and VTF control, the linear quadratic regulator control (LQR) and model predictive control (MPC) are compared in Table 1. The memory utilization of different controllers are the same value. However, the CPU utilization are much different.
The LQR approach has the maximum CPU utilization (as high as 30.9%) when its maximum iteration is set to 150. The CPU utilization of MPC approach is 7.2% even though the prediction domain is only 10 steps. The CPU utilization of VTF approach is only 1.9%, which is even smaller than that of PID control.
From the above analysis, it can be concluded that the steering control based on VTF approach has a robustness in fitting different working conditions without increasing the steering load. At the same time, the real-time performance is also excellent.
VI. Conclusion
Inspired by the ACP approach, this paper presented the IoT based parallel mining as a new solution for end-to-end autonomous mining operations, to improve the safety of personnel and equipment, reduce the cost of mining operation and increase production rate fundamentally. As the four main parts of parallel mining, the M&C center, the autonomous transportation platform of truck, the semi-autonomous mining/shovelling platform, as well as the remote take-over platform, were discussed in detail. The key technological advances achieved in the construction of parallel mining, including the network communication, virtual parallel mine construction, mining environment perception, collaborative decision-making, unmanned mining equipment control, as well as parallel taking-over and remote control, were presented and discussed as examples.
The case study for performance evaluation of IoT based parallel mining were evaluated. From the study results, it can be concluded that the IoT-oriented approach has obvious advantage in perception, decision-making, planning and control. IoT based parallel mining has excellent real-time performance for decision-making and control, and could improves the stability, efficiency and safety of the autonomous trucks.
Our on-going work is devoted to the improvement and implementation of intelligent scheduling algorithm. Future work involves the statistical analysis of operation data of several unmanned mines, to evaluate the advantages of IoT based parallel mining solutions in reducing operation costs, improving efficiency and enhancing safety.
References
M. A. Razzaque, M. Milojevic-Jevric, A. Palade, and S. Clarke, “Middleware for internet of things: A survey,” IEEE Internet of Things Journal, vol. 3, no. 1, pp. 70–95, 2016.
Paul J. A. Lever and Fei-Yue Wang, “Intelligent Excavator Control System for Lunar Mining System”, Journal of Aerospace Engineering, 1995, Vol. 8, No. 1, pp. 16-24.
Paul Lever, Fei-Yue Wang, “Intelligent Excavation Control for a Lunar Mining System”, Proc. of 4th International Conference on Robotics for Challenging Environments, Albuquerque, NM, 1994, pp. 97-105.
Levelling the Playing Field: The Mining Association of Canada. Report. April,2015.
Mining Feature: Fatigue Management for Haul Truck Drivers: A Low-Tech Solution for a Small Mine". Office of Mine Safety and Health Research, National Institute for Occupational Safety and Health. 28 October 2014.
Nunes, Ashley, Bryan Reimer, and Joseph F. Coughlin. "People must retain control of autonomous vehicles." Nature 556 (2018): 169-171.
Contreras J, Zeadally S, Guerrero-Ibanez J A. "Internet of vehicles: Architecture, protocols, and security." IEEE Internet of Things Journal (2017).
"Map of unmanned enterprises in China, 2018 Edition". Zhigu news. Available: http://zhigu.news.cn/2018-09/29/c_129963390.htm.
"2017 RBR50 List Names the Leading Robotics Companies of the Year ". Robotics Business Review. Retrieved 2017. Available: https://www. roboticsbusinessreview.com/companies/?companyType=rbr_50.
Paul J.A.Lever, Fei-Yue Wang, D.Q. Chen, “A Fuzzy Control System for an Automated Mining Excavator”, Proceedings of 1994 IEEE International Conference on Robotics and Automation, San Diego, CA, May. 8-13, 1994, Vol. 4, pp. 3284-3289.
Lever, Paul JA, et al. "Autonomous robotic mining excavation using fuzzy logic and neural networks." Journal of Intelligent & Fuzzy Systems 3.1 (1995): 31-42.
X. Chen, M. He. "Framework system and key technology of intelligent mine construction." Coal Science and Technology, 2018, 46(2).
Parreira, Juliana, and John Meech. "Autonomous haulage systems–justification and opportunity." International Conference on Autonomous and Intelligent Systems. Springer, Berlin, Heidelberg, 2011.
Mining technology - CAT® minestar. Available: https://www.cat.com/ en_US/by-industry/mining/mining-solutions/technology-new.html
Glass labyrinth: 38-tonne automated loader. Available: https://www.rocktechnology.sandvik/en/
Volvo tests a self-driving truck in a swedish mine, Blows Our Minds. Available: https://www.guidingtech.com/61345/volvo-tests-self-driving -truck-swedish-mine/
Min Zuo. The "Application of Parallel System Theory in Modern Mine Managemnet. " METAL MINE, 2011, 40(9):131-133.
Wang F Y. "Parallel system methods for management and control of complex systems. " Control and Decision, 2004, 19(5): 485-489.
Bai T., Wang S., Shen Z., et al., "Parallel Robotics and Parallel Unmanned Systems: Framework, Structure, Process, Platform and Applications. " Acta Automatica Sinica, 2017, 43(2): 161-175.
Wang F Y. "Software-defined Systems and Knowledge Automation: a Parallel Paradigm Shift from Newton to Merton." Acta Automatica Sinica, 2015, 41(1): 1-8.
Wang F Y. "Parallel Control: A Method for Data-Driven and Compu- tational Control." Acta Automatica Sinica, 2013, 39(4): 293-302.
Wang F Y. "Artificial Societies, Computational Expermients, and Parallel Systems:A Discussion on Computational Theory of Complex Social- Economic Systems." Complex Systems and Complexity Science, 2004, 1(4): 25-35.
Fortino, Giancarlo, et al. "Towards a development methodology for smart object-oriented IoT systems: A metamodel approach." 2015 IEEE international conference on systems, man, and cybernetics. IEEE, 2015.
Fortino, Giancarlo, et al. "Agent-oriented cooperative smart objects: From IoT system design to implementation." IEEE Transactions on Systems, Man, and Cybernetics: Systems48.11 (2017): 1939-1956.
Chen, Min, et al. "A Dynamic Service Migration Mechanism in Edge Cognitive Computing." ACM Transactions on Internet Technology (TOIT) 19.2 (2019): 30.
Wang F Y. "Computational Theory and Method on Complex System." China Basic Science, 2004, 6(5): 3-10.
Wang F Y, ZHENG N N, CAO D P, et al. "Parallel driving in CPSS: a unified approach for transport automation and vehicle intelligence." IEEE/CAA Journal of Automatica Sinica, 2017, 4(4): 577-587.
Wang F Y. "The emergence of intelligent enterprises: from CPS to CPSS". IEEE Intelligent Systems, 2010, 25(4): 85-88.
Paul Lever, Fei-Yue Wang and X. Shi, “An Intelligent Task Control System for Dynamic Mining Environments”, AIME Transactions on Mining, Metallurgy and Exploration, 1994, Vol. 174, No. 1, pp. 165-174.
P. J. A. Lever, Fei-Yue Wang, X. Shi, D. Chen, “A fuzzy-behavior-based approach for controlling mining excavator bucket/rock interactions”, Proceedings of 1994 IEEE Industry Applications Society Annual Meeting, Denver, CO, USA, Oct. 2-6, 1994, Vol. 3, pp. 2126-2133.
LIU T., YU H., TIAN B., et al. "Intelligent command and control systems for intelligent vehicle: primary methods and systemic construction." Journal of Command and Control, 2018, 4(1): 22-31.
Li, Li, et al. "Parallel testing of vehicle intelligence via virtual-real interaction." Sci. Robot 4 (2019).
Huile Xu, Yi Zhang, Li Li, Weixia Li, "Cooperative driving at unsignalized intersections using tree search," https://arxiv.org/abs/1902.01024
Li Li, Fei-Yue Wang, "Cooperative driving at blind crossings using intervehicle communication," IEEE Transactions on Vehicular Technology, vol. 55, no. 6, pp. 1712-1724, 2006.
Gao, Y., Shen, Y., Zhang, W., Wang, H. et al., “Robust Path Tracking Control for Autonomous Heavy Vehicles,” SAE Technical Paper 2018-01-1082, 2018, doi:10.4271/2018-01-1082.
Gao, Y., Shen, Y., Cao, D., "Path-Following Control by Dynamic Virtual Terrain Field for Articulated Steer Vehicles" Vehicle System Dynamics, https://doi.org/10.1080/00423114.2019.1648837.
M. Darms, C. R. Baker, P. E. Rybski, and C. Urmson, “Vehicle detection and tracking for the Urban Challenge Vehicle Detection and Trackin g for the Urban Challenge,” vol. 10,no. 3, pp. 57 - 67, 2008.
Yang F., Zhu Z. , Dong X., et al. "Real-time dynamic obstacle detection and tracking using 3D Lidar." Journal of Zhejiang University (Engineering Science), 2012, 46(9):1565 - 1571.
Ester, Martin, et al. "A density-based algorithm for discovering clusters in large spatial databases with noise." Kdd. Vol. 96. No. 34. 1996.
S . Thrun , M . Montemerlo, and A. Aron, "Probabilistic Terrain Analysis For High-Speed Desert Driving." Robotics: Science and Systems. 2006.
Y. Ye, L. Fu, and B. Li, “Object Detection and Tracking Using Multi - layer Laser for Autonomous Urban Driving,” ITSC, 2016 IEEE 19th International Conference on. IEEE, 2016.
Matti Lehtomäki, et al. "Object Classification and Recognition From Mobile Laser Scanning Point Clouds in a Road Environment." IEEE Transactions on Geoscience & Remote Sensing 54.2(2016):1226-1239.
Yue Meng, Li Li, Fei-Yue Wang, Keqiang Li, Zhiheng Li, "Analysis of cooperative driving strategies for nonsignalized intersections," IEEE Transactions on Vehicular Technology, vol. 67, no. 4, pp. 2900-2911, 2018.
Kritayakirana, Krisada, and J. Christian Gerdes. Autonomous vehicle control at the limits of handling. International Journal of Vehicle Autonomous Systems 10.4 (2012): 271-296.
https://blog.sciencenet.cn/blog-2374-1211246.html
上一篇：[转载]IEEE/CAA Journal of Automatica Sinica被SCI收录
下一篇：[转载]【CAA新闻】2019平行智能大会在北京成功举办
